Re: [fpc-pascal] String conversions

[Zoë Peterson]

On 6/28/2019 3:07 PM, Ryan Joseph wrote:

On Jun 28, 2019, at 3:40 PM, Martok  wrote:
So, I would expect (and FastMM has codepaths for that), that repeated
reallocations cause some form of "over-allocating" growth and most of the
individual "+1" reallocs will be essentially no-ops.


Interesting. How could we test this to see if it’s true?


From FastMM4's readme:

https://github.com/pleriche/FastMM4/blob/master/README.md

"Intelligent reallocations. Avoids slow memory move operations through 
not performing unneccesary downsizes and by having a minimum percentage 
block size growth factor when an in-place block upsize is not possible."


Allocations within certain ranges are all lumped together based on fixed 
block sizes (e.g., 15-24 bytes allocations are all rounded up to 24 
bytes).  It both reduces fragmentation and helps with resize performance 
(there is other resize-specific logic as well).


String allocations and performance factored heavily into the 
benchmarking for the FastCode competition that lead to FastMM4.  You 
should do your own benchmarking vs the system allocator though, to see 
if it provides an improvement, especially if you're doing extensive 
multithreading.  We've been using FastMM with FPC on macOS and Linux for 
about 4 years now and are very happy with it.


--
Zoë Peterson
Scooter Software

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] String conversions

[Ryan Joseph]

> On Jun 28, 2019, at 3:40 PM, Martok  wrote:
> 
> So, I would expect (and FastMM has codepaths for that), that repeated
> reallocations cause some form of "over-allocating" growth and most of the
> individual "+1" reallocs will be essentially no-ops.

Interesting. How could we test this to see if it’s true?

Regards,
Ryan Joseph

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] String conversions

[Martok]

> Yes indeed. FPC already has an overwhelming amount of string types. As I said 
> though a SetCapacity option for growing would be nice (for dynamic arrays 
> also) because += is such a common operation. As it stands I often don’t use 
> dynamic arrays (and now ansistring) because of growing limitations, which is 
> a shame really.

Just chiming in: this should be handled by the memory manager.
"A chunk of memory gets grown repeatedly" is actually a fairly common workload,
not just for arrays, but also things like MemoryStreams or List classes.

So, I would expect (and FastMM has codepaths for that), that repeated
reallocations cause some form of "over-allocating" growth and most of the
individual "+1" reallocs will be essentially no-ops.

-- 
Regards,
Martok

Ceterum censeo b32079 esse sanandam.

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] String conversions

[Sven Barth via fpc-pascal]

Am 27.06.2019 um 07:10 schrieb Sven Barth:

Am 26.06.2019 um 23:36 schrieb Ryan Joseph:



On Jun 26, 2019, at 3:28 PM, Ben Grasset  wrote:

I think Ryan probably meant his own custom types. And certainly, you 
can do some interesting stuff with operator overloading that mostly 
avoids the normal AnsiString overhead. Here's an example.
Yes indeed. FPC already has an overwhelming amount of string types. 
As I said though a SetCapacity option for growing would be nice (for 
dynamic arrays also) because += is such a common operation. As it 
stands I often don’t use dynamic arrays (and now ansistring) because 
of growing limitations, which is a shame really.
No. That would mean an additional entry in the metadata of strings and 
arrays and those are already big enough in my opinion compared to the 
data they're handling.
If one needs capacity based handling for these types then one should 
use helper types.

Note: with which I mean types like TStringBuilder, not type helpers.

Regards,
Sven
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] String conversions

[Sven Barth via fpc-pascal]

Am 26.06.2019 um 23:36 schrieb Ryan Joseph:



On Jun 26, 2019, at 3:28 PM, Ben Grasset  wrote:

I think Ryan probably meant his own custom types. And certainly, you can do 
some interesting stuff with operator overloading that mostly avoids the normal 
AnsiString overhead. Here's an example.

Yes indeed. FPC already has an overwhelming amount of string types. As I said 
though a SetCapacity option for growing would be nice (for dynamic arrays also) 
because += is such a common operation. As it stands I often don’t use dynamic 
arrays (and now ansistring) because of growing limitations, which is a shame 
really.
No. That would mean an additional entry in the metadata of strings and 
arrays and those are already big enough in my opinion compared to the 
data they're handling.
If one needs capacity based handling for these types then one should use 
helper types.


Regards,
Sven
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] String conversions

[Ben Grasset]

On Wed, Jun 26, 2019 at 5:36 PM Ryan Joseph  wrote:

> Yes indeed. FPC already has an overwhelming amount of string types. As I
> said though a SetCapacity option for growing would be nice (for dynamic
> arrays also) because += is such a common operation. As it stands I often
> don’t use dynamic arrays (and now ansistring) because of growing
> limitations, which is a shame really.
>

Well, I think the idea is that stuff like capacity is what you'd have / do
have in actual containers built *around* strings and / or dynamic arrays,
as they're both just built in primitives.
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] String conversions

[Ryan Joseph]

> On Jun 26, 2019, at 3:28 PM, Ben Grasset  wrote:
> 
> I think Ryan probably meant his own custom types. And certainly, you can do 
> some interesting stuff with operator overloading that mostly avoids the 
> normal AnsiString overhead. Here's an example.

Yes indeed. FPC already has an overwhelming amount of string types. As I said 
though a SetCapacity option for growing would be nice (for dynamic arrays also) 
because += is such a common operation. As it stands I often don’t use dynamic 
arrays (and now ansistring) because of growing limitations, which is a shame 
really.

Regards,
Ryan Joseph

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] String conversions

[Ben Grasset]

On Wed, Jun 26, 2019 at 3:28 PM Ben Grasset  wrote:

> I think Ryan probably meant his own custom types. And certainly, you can
> do some interesting stuff with operator overloading that mostly avoids the
> normal AnsiString overhead. Here's an example.
> 
>

Messed up the "Reversed" function a little bit in that example. Fixed
version.

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] String conversions

[Ben Grasset]

On Wed, Jun 26, 2019 at 5:48 AM Michael Van Canneyt 
wrote:

> I second that.
>
> Michael.
>

I think Ryan probably meant his own custom types. And certainly, you can do
some interesting stuff with operator overloading that mostly avoids the
normal AnsiString overhead. Here's an example.

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] String conversions

[Michael Van Canneyt]

On Wed, 26 Jun 2019, Marco van de Voort wrote:



Op 25/06/2019 om 21:58 schreef Ryan Joseph:
An exponential growing function applied to all strings would have many 
worse case behaviours, where it would eat up heaps of memory for nothing. 
This is why it is better to have the programmer indicate it is a growing 
string some how (or have a separate type, like TStringBuilder mentioned by 
Jonas)
It would be nice if ansistring had a SetCapacity function to accompany 

SetLength but in leu of that I think making a new type is the best idea.

I think we have enough string types 
http://www.stack.nl/~marcov/delphistringtypes.txt


TStringbuilder is fine . Not every problem needs new syntax or types to 
solve.


I second that.

Michael.
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] String conversions

[Marco van de Voort]

Op 25/06/2019 om 21:58 schreef Ryan Joseph:

An exponential growing function applied to all strings would have many worse 
case behaviours, where it would eat up heaps of memory for nothing. This is why 
it is better to have the programmer indicate it is a growing string some how 
(or have a separate type, like TStringBuilder mentioned by Jonas)
It would be nice if ansistring had a SetCapacity function to accompany 
SetLength but in leu of that I think making a new type is the best idea.


I think we have enough string types 
http://www.stack.nl/~marcov/delphistringtypes.txt


TStringbuilder is fine . Not every problem needs new syntax or types to 
solve.


___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] String conversions

[Ryan Joseph]

> On Jun 25, 2019, at 3:55 PM, Marco van de Voort  
> wrote:
> 
> An exponential growing function applied to all strings would have many worse 
> case behaviours, where it would eat up heaps of memory for nothing. This is 
> why it is better to have the programmer indicate it is a growing string some 
> how (or have a separate type, like TStringBuilder mentioned by Jonas)

It would be nice if ansistring had a SetCapacity function to accompany 
SetLength but in leu of that I think making a new type is the best idea.

Regards,
Ryan Joseph

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] String conversions

[Marco van de Voort]

Op 2019-06-25 om 21:12 schreef Ryan Joseph:



On Jun 25, 2019, at 2:16 PM, Jonas Maebe  wrote:

It has to allocate a new ansistring.

Another question: if I do "s += c” will the string reallocate memory for 1 
character or is there an exponential growing function? If it does grow by one 
character at a time how can I reserve a certain capacity in advance?
An exponential growing function applied to all strings would have many 
worse case behaviours, where it would eat up heaps of memory for 
nothing. This is why it is better to have the programmer indicate it is 
a growing string some how (or have a separate type, like TStringBuilder 
mentioned by Jonas)

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] String conversions

[Jonas Maebe]

On 25/06/2019 21:12, Ryan Joseph wrote:
> 
> Another question: if I do "s += c” will the string reallocate memory for 1 
> character or is there an exponential growing function? If it does grow by one 
> character at a time how can I reserve a certain capacity in advance?

Strings always contain the exact amount of memory required for their length.

If you need to do a lot of concatenations, you can use the
TStringBuilder class from sysutils for better performance.


Jonas
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] String conversions

[Ryan Joseph]

> On Jun 25, 2019, at 2:16 PM, Jonas Maebe  wrote:
> 
> It has to allocate a new ansistring.

Another question: if I do "s += c” will the string reallocate memory for 1 
character or is there an exponential growing function? If it does grow by one 
character at a time how can I reserve a certain capacity in advance?

Regards,
Ryan Joseph

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] String conversions

[Ryan Joseph]

> On Jun 25, 2019, at 2:16 PM, Jonas Maebe  wrote:
> 
> On 25/06/2019 20:15, Ryan Joseph wrote:
>> I’m making some string helper functions and using ansistring as inputs (in 
>> case there are longer strings than 255 chars). What happens when you pass a 
>> short string as an ansistring? Does the compiler have to allocate a new 
>> ansistring or can it do some smart optimization? 
> 
> It has to allocate a new ansistring.

Ouch. That means I need to make lots of overloads. This is a prime candidate 
for generics then but I’m still waiting for my 
“implicitfunctionspecializations” modeswitch patch to be accepted.

Regards,
Ryan Joseph

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] String conversions

[Jonas Maebe]

On 25/06/2019 20:15, Ryan Joseph wrote:
> I’m making some string helper functions and using ansistring as inputs (in 
> case there are longer strings than 255 chars). What happens when you pass a 
> short string as an ansistring? Does the compiler have to allocate a new 
> ansistring or can it do some smart optimization? 

It has to allocate a new ansistring.


Jonas
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

[fpc-pascal] String conversions

[Ryan Joseph]

I’m making some string helper functions and using ansistring as inputs (in case 
there are longer strings than 255 chars). What happens when you pass a short 
string as an ansistring? Does the compiler have to allocate a new ansistring or 
can it do some smart optimization? 

Regards,
Ryan Joseph

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
https://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal