Re: Most performant way of converting int to string

[Ivan Kazmenko via Digitalmars-d-learn]

On Tuesday, 22 December 2015 at 19:50:28 UTC, Daniel Kozák wrote:

V Tue, 22 Dec 2015 18:39:16 +
Ivan Kazmenko via Digitalmars-d-learn
 napsáno:
Does DMD, or Phobos function to!(string), do anything like 
that? The number of possible bases is not large anyway.  I've 
heard major C/C++ compilers do that, but have not looked for a 
proof myself.


Yes, IIRC, all D compilers should be able to change modulus and 
division to few additions, subtractions and bit shifts if base 
is known at compile time.


And phobos use this: 
https://github.com/D-Programming-Language/phobos/pull/1452


So, in a few common cases (bases 10 and 2, 4, 8, 16), you convert 
a runtime argument into a compile-time argument for the 
implementation function, and there, division is optimized at 
compile time.  Nice!

Re: Most performant way of converting int to string

[Jakob Ovrum via Digitalmars-d-learn]

On Tuesday, 22 December 2015 at 17:23:11 UTC, Andrew Chapman 
wrote:

On Tuesday, 22 December 2015 at 17:18:16 UTC, cym13 wrote:
On Tuesday, 22 December 2015 at 17:15:27 UTC, Andrew Chapman 
wrote:
Sorry if this is a silly question but is the to! method from 
the conv library the most efficient way of converting an 
integer value to a string?


e.g.
string s = to!string(100);

I'm seeing a pretty dramatic slow down in my code when I use 
a conversion like this (when looped over 10 million 
iterations for benchmarking).


Cheers!


Out of curiosity a slow down compared to what? No conversion 
at all?


Yeah, if I include a simple conversion in my loop:

for({int i; i = 0;} i < num; i++) {
//string s = to!string(i);
Customer c = Customer(i, "Customer", 
"", i * 2);

string result = objS.serialize(c);
}

If I uncomment the "string s" line I'm seeing a 20% increase in 
running time, which given what's going on the rest of the code 
is quite surprising.  I've tried compiling with both dmd and 
ldc2 - it's the same under both.


Cheers.


Dynamic memory allocation is expensive. If the string is 
short-lived, allocate it on the stack:


enum maxDigits = to!string(ulong.max).length;
foreach(i; 0 .. num) {
char[maxDigits] buffer = void;
auto c = Customer(sformat(buffer[], "%s", i));
string result = objS.serialize(c);
}

Note that this is unsafe if the string (sformat's return value) 
outlives the loop iteration, as this is `buffer`'s scope.

Re: Most performant way of converting int to string

[Jakob Ovrum via Digitalmars-d-learn]

On Wednesday, 23 December 2015 at 11:21:32 UTC, Jakob Ovrum wrote:
Dynamic memory allocation is expensive. If the string is 
short-lived, allocate it on the stack:


See also std.conv.toChars[1] for stringifying lazily/on-demand.

http://dlang.org/phobos/std_conv#toChars

Re: Most performant way of converting int to string

[Ali Çehreli via Digitalmars-d-learn]

On 12/23/2015 02:29 PM, Andrew Chapman wrote:

> string v = toChars!(16,char,LetterCase.lower)(i);
>
> to convert an integer to Hex for example, but the compiler wasn't happy
> with it.  How would I convert an int to a string using this?

I had to squint at the error message to understand that the 16 
specialization for radix requires that the value is unsigned. The 
following works:


import std.conv;
import std.algorithm;

void main() {
assert(1234u// <-- unsigned
   .toChars!(16, char, LetterCase.lower)
   .equal("4d2"));
}

Ali

Re: Most performant way of converting int to string

[Andrew Chapman via Digitalmars-d-learn]

On Wednesday, 23 December 2015 at 11:46:37 UTC, Jakob Ovrum wrote:
On Wednesday, 23 December 2015 at 11:21:32 UTC, Jakob Ovrum 
wrote:
Dynamic memory allocation is expensive. If the string is 
short-lived, allocate it on the stack:


See also std.conv.toChars[1] for stringifying lazily/on-demand.

http://dlang.org/phobos/std_conv#toChars


Thanks Jakob!  I did try toChars but I couldn't quite figure out 
a syntax of calling it that the compiler was happy with.  From 
memory I tried things along the lines of:


string v = toChars!(16,char,LetterCase.lower)(i);

to convert an integer to Hex for example, but the compiler wasn't 
happy with it.  How would I convert an int to a string using this?


Cheers.

Re: Most performant way of converting int to string

[Minas Mina via Digitalmars-d-learn]

On Wednesday, 23 December 2015 at 22:29:31 UTC, Andrew Chapman 
wrote:
On Wednesday, 23 December 2015 at 11:46:37 UTC, Jakob Ovrum 
wrote:
On Wednesday, 23 December 2015 at 11:21:32 UTC, Jakob Ovrum 
wrote:
Dynamic memory allocation is expensive. If the string is 
short-lived, allocate it on the stack:


See also std.conv.toChars[1] for stringifying lazily/on-demand.

http://dlang.org/phobos/std_conv#toChars


Thanks Jakob!  I did try toChars but I couldn't quite figure 
out a syntax of calling it that the compiler was happy with.  
From memory I tried things along the lines of:


string v = toChars!(16,char,LetterCase.lower)(i);

to convert an integer to Hex for example, but the compiler 
wasn't happy with it.  How would I convert an int to a string 
using this?


Cheers.


I haven't tested it, but:
`toChars` doesn't return a string -- that's the whole point :)
It returns a range, so you have to call it something like:
auto v = toChars!(16,char,LetterCase.lower)(i);

Re: Most performant way of converting int to string

[Daniel Kozák via Digitalmars-d-learn]

V Tue, 22 Dec 2015 18:39:16 +
Ivan Kazmenko via Digitalmars-d-learn
 napsáno:

> On Tuesday, 22 December 2015 at 18:11:24 UTC, rumbu wrote:
> > On Tuesday, 22 December 2015 at 17:15:27 UTC, Andrew Chapman 
> > wrote:  
> >> Sorry if this is a silly question but is the to! method from 
> >> the conv library the most efficient way of converting an 
> >> integer value to a string?
> >>
> >> e.g.
> >> string s = to!string(100);
> >>
> >> I'm seeing a pretty dramatic slow down in my code when I use a 
> >> conversion like this (when looped over 10 million iterations 
> >> for benchmarking).
> >>
> >> Cheers!  
> >
> > Converting numbers to string involves the most expensive known 
> > two operations : division and modulus by 10.  
> 
> When the base is known in advance, division and modulus can be 
> substituted by a few additions, subtractions and bit shifts.  For 
> example, the Hacker's Delight book has a chapter dedicated to 
> that, as well as a freely available additional chapter 
> (www.hackersdelight.org/divcMore.pdf).
> 
> Does DMD, or Phobos function to!(string), do anything like that?  
> The number of possible bases is not large anyway.  I've heard 
> major C/C++ compilers do that, but have not looked for a proof 
> myself.

Yes, IIRC, all D compilers should be able to change modulus and
division to few additions, subtractions and bit shifts if base is known
at compile time.

And phobos use this:
https://github.com/D-Programming-Language/phobos/pull/1452

Re: Most performant way of converting int to string

[Daniel Kozák via Digitalmars-d-learn]

V Tue, 22 Dec 2015 18:11:24 +
rumbu via Digitalmars-d-learn 
napsáno:

> On Tuesday, 22 December 2015 at 17:15:27 UTC, Andrew Chapman 
> wrote:
> > Sorry if this is a silly question but is the to! method from 
> > the conv library the most efficient way of converting an 
> > integer value to a string?
> >
> > e.g.
> > string s = to!string(100);
> >
> > I'm seeing a pretty dramatic slow down in my code when I use a 
> > conversion like this (when looped over 10 million iterations 
> > for benchmarking).
> >
> > Cheers!  
> 
> Converting numbers to string involves the most expensive known 
> two operations : division and modulus by 10.

No, IIRC few months or maybe years I have optimize this so it does not
use division and modulus

Re: Most performant way of converting int to string

[rumbu via Digitalmars-d-learn]

On Tuesday, 22 December 2015 at 17:15:27 UTC, Andrew Chapman 
wrote:
Sorry if this is a silly question but is the to! method from 
the conv library the most efficient way of converting an 
integer value to a string?


e.g.
string s = to!string(100);

I'm seeing a pretty dramatic slow down in my code when I use a 
conversion like this (when looped over 10 million iterations 
for benchmarking).


Cheers!


Converting numbers to string involves the most expensive known 
two operations : division and modulus by 10.

Re: Most performant way of converting int to string

[Daniel Kozák via Digitalmars-d-learn]

V Tue, 22 Dec 2015 17:15:27 +
Andrew Chapman via Digitalmars-d-learn
 napsáno:

> Sorry if this is a silly question but is the to! method from the 
> conv library the most efficient way of converting an integer 
> value to a string?
> 
> e.g.
> string s = to!string(100);
> 
> I'm seeing a pretty dramatic slow down in my code when I use a 
> conversion like this (when looped over 10 million iterations for 
> benchmarking).
> 
> Cheers!
> 
> 

This is OK and expected :).

Re: Most performant way of converting int to string

[Daniel Kozák via Digitalmars-d-learn]

V Tue, 22 Dec 2015 20:52:07 +
rumbu via Digitalmars-d-learn 
napsáno:

> On Tuesday, 22 December 2015 at 19:45:46 UTC, Daniel Kozák wrote:
> > V Tue, 22 Dec 2015 18:11:24 +
> > rumbu via Digitalmars-d-learn 
> > 
> > napsáno:
> >  
> >> On Tuesday, 22 December 2015 at 17:15:27 UTC, Andrew Chapman 
> >> wrote:  
> >> > Sorry if this is a silly question but is the to! method from 
> >> > the conv library the most efficient way of converting an 
> >> > integer value to a string?
> >> >
> >> > e.g.
> >> > string s = to!string(100);
> >> >
> >> > I'm seeing a pretty dramatic slow down in my code when I use 
> >> > a conversion like this (when looped over 10 million 
> >> > iterations for benchmarking).
> >> >
> >> > Cheers!  
> >> 
> >> Converting numbers to string involves the most expensive known 
> >> two operations : division and modulus by 10.  
> >
> > No, IIRC few months or maybe years I have optimize this so it 
> > does not use division and modulus  
> 
> It's using division and modulus, as expected:
> 
> https://github.com/D-Programming-Language/phobos/blob/master/std/conv.d#L5529
> 
> If the compiler is smart enough, maybe it will replace this by 
> well known multiplication trick.

Yes compiler is smart enought so it will not use division and modulus
as I say before :)

Re: Most performant way of converting int to string

[Andrew Chapman via Digitalmars-d-learn]

On Tuesday, 22 December 2015 at 18:11:24 UTC, rumbu wrote:
On Tuesday, 22 December 2015 at 17:15:27 UTC, Andrew Chapman 
wrote:
Sorry if this is a silly question but is the to! method from 
the conv library the most efficient way of converting an 
integer value to a string?


e.g.
string s = to!string(100);

I'm seeing a pretty dramatic slow down in my code when I use a 
conversion like this (when looped over 10 million iterations 
for benchmarking).


Cheers!


Converting numbers to string involves the most expensive known 
two operations : division and modulus by 10.


Cool thanks, so essentially it's unavoidable - I have a 
background in PHP programming where we don't really get exposed 
to memory allocation, conversions etc.  Good to learn.  Cheers.

Re: Most performant way of converting int to string

[rumbu via Digitalmars-d-learn]

On Tuesday, 22 December 2015 at 20:52:07 UTC, rumbu wrote:
On Tuesday, 22 December 2015 at 19:45:46 UTC, Daniel Kozák 
wrote:

V Tue, 22 Dec 2015 18:11:24 +
rumbu via Digitalmars-d-learn 


napsáno:

On Tuesday, 22 December 2015 at 17:15:27 UTC, Andrew Chapman 
wrote:
> Sorry if this is a silly question but is the to! method 
> from the conv library the most efficient way of converting 
> an integer value to a string?

>
> e.g.
> string s = to!string(100);
>
> I'm seeing a pretty dramatic slow down in my code when I 
> use a conversion like this (when looped over 10 million 
> iterations for benchmarking).

>
> Cheers!

Converting numbers to string involves the most expensive 
known two operations : division and modulus by 10.


No, IIRC few months or maybe years I have optimize this so it 
does not use division and modulus


It's using division and modulus, as expected:

https://github.com/D-Programming-Language/phobos/blob/master/std/conv.d#L5529

If the compiler is smart enough, maybe it will replace this by 
well known multiplication trick.


Tested it, I was right:

No optimizations: two divisions, one for the modulus and one for 
the division itself

Optimized: multiplication by 0xCCCD trick.

http://imgur.com/a/lHeHe

Re: Most performant way of converting int to string

[H. S. Teoh via Digitalmars-d-learn]

On Tue, Dec 22, 2015 at 08:54:35PM +0100, Daniel Kozák via Digitalmars-d-learn 
wrote:
> V Tue, 22 Dec 2015 09:43:00 -0800
> "H. S. Teoh via Digitalmars-d-learn"
>  napsáno:
> 
> > On Tue, Dec 22, 2015 at 05:23:11PM +, Andrew Chapman via
> > Digitalmars-d-learn wrote: [...]
> > > for({int i; i = 0;} i < num; i++) {
> > > //string s = to!string(i);
> > > Customer c = Customer(i, "Customer", "", i
> > > * 2); string result = objS.serialize(c);
> > > }
> > > 
> > > If I uncomment the "string s" line I'm seeing a 20% increase in
> > > running time, which given what's going on the rest of the code is
> > > quite surprising.  I've tried compiling with both dmd and ldc2 -
> > > it's the same under both.  
> > [...]
> > 
> > I wonder if the slowdown is caused by GC collection cycles (because
> > calling to!string will allocate, and here you're making a very large
> > number of small allocations, which is known to cause GC performance
> > issues).
> > 
> > Try inserting this before the loop:
> > 
> > import core.memory;
> > GC.disable();
> > 
> > Does this make a difference in the running time?
> > 
> > 
> > T
> > 
> This would  not help. It would probably be worse.

I was not suggesting this as a solution, it's merely a way to determine
whether the performance issue is GC-related.


T

-- 
"Hi." "'Lo."

Re: Most performant way of converting int to string

[Daniel Kozák via Digitalmars-d-learn]

V Tue, 22 Dec 2015 21:10:54 +
rumbu via Digitalmars-d-learn 
napsáno:

> On Tuesday, 22 December 2015 at 20:52:07 UTC, rumbu wrote:
> > On Tuesday, 22 December 2015 at 19:45:46 UTC, Daniel Kozák 
> > wrote:  
> >> V Tue, 22 Dec 2015 18:11:24 +
> >> rumbu via Digitalmars-d-learn 
> >> 
> >> napsáno:
> >>  
> >>> On Tuesday, 22 December 2015 at 17:15:27 UTC, Andrew Chapman 
> >>> wrote:  
> >>> > Sorry if this is a silly question but is the to! method 
> >>> > from the conv library the most efficient way of converting 
> >>> > an integer value to a string?
> >>> >
> >>> > e.g.
> >>> > string s = to!string(100);
> >>> >
> >>> > I'm seeing a pretty dramatic slow down in my code when I 
> >>> > use a conversion like this (when looped over 10 million 
> >>> > iterations for benchmarking).
> >>> >
> >>> > Cheers!  
> >>> 
> >>> Converting numbers to string involves the most expensive 
> >>> known two operations : division and modulus by 10.  
> >>
> >> No, IIRC few months or maybe years I have optimize this so it 
> >> does not use division and modulus  
> >
> > It's using division and modulus, as expected:
> >
> > https://github.com/D-Programming-Language/phobos/blob/master/std/conv.d#L5529
> >
> > If the compiler is smart enough, maybe it will replace this by 
> > well known multiplication trick.  
> 
> Tested it, I was right:
> 
> No optimizations: two divisions, one for the modulus and one for 
> the division itself
> Optimized: multiplication by 0xCCCD trick.
> 
> http://imgur.com/a/lHeHe
> 
> 

Yes this is expected :).

Re: Most performant way of converting int to string

[Steven Schveighoffer via Digitalmars-d-learn]

On 12/22/15 12:15 PM, Andrew Chapman wrote:

Sorry if this is a silly question but is the to! method from the conv
library the most efficient way of converting an integer value to a string?

e.g.
string s = to!string(100);

I'm seeing a pretty dramatic slow down in my code when I use a
conversion like this (when looped over 10 million iterations for
benchmarking).

Cheers!


Does it need to be string? What about a mutable char buffer?

If you are allocating and throwing away strings in a tight loop, simply 
reusing the buffer could save a ton of time.


Look into std.format

-Steve

Re: Most performant way of converting int to string

[Daniel Kozak via Digitalmars-d-learn]

On Tuesday, 22 December 2015 at 18:27:12 UTC, cym13 wrote:

...
I don't think there is anything in the standard
library that would really help here as (if I read it correctly) 
it is mainly
because of the conversion from ranges to arrays that this code 
is slow.


Yes, it has been faster in past, when I last check and optimeze 
to!string. But because of @nogc it has been rewriten to Range 
base version which is slower. In this case it makes sense to add 
special version without @nogc (to!string will allocate memory 
anyway) which just use dup on buffer instead of make Range.


I will probably make pull request tomorrow.

Re: Most performant way of converting int to string

[Ali Çehreli via Digitalmars-d-learn]

On 12/22/2015 10:15 AM, Andrew Chapman wrote:
> On Tuesday, 22 December 2015 at 18:11:24 UTC, rumbu wrote:

>> Converting numbers to string involves the most expensive known two
>> operations : division and modulus by 10.
>
> Cool thanks, so essentially it's unavoidable

There is hope. :)

> I have a background in PHP programming where we don't really get 
exposed to

> memory allocation, conversions etc.

You will love Andrei's recently-released presentations from code::dive 
in November 2015:


  Writing Fast Code I: https://www.youtube.com/watch?v=ph7FP0LnmcA

  Writing Fast Code II: https://www.youtube.com/watch?v=3_FXy3cT5C8

  Three Cool Things about D: https://youtube.com/watch?v=FdpaBHyQNco

The first one has an example of optimizing uint64ToAscii() at around 
1:16:30:



https://www.youtube.com/watch?feature=player_detailpage=ph7FP0LnmcA#t=4604

Ali

Re: Most performant way of converting int to string

[Ivan Kazmenko via Digitalmars-d-learn]

On Tuesday, 22 December 2015 at 18:11:24 UTC, rumbu wrote:
On Tuesday, 22 December 2015 at 17:15:27 UTC, Andrew Chapman 
wrote:
Sorry if this is a silly question but is the to! method from 
the conv library the most efficient way of converting an 
integer value to a string?


e.g.
string s = to!string(100);

I'm seeing a pretty dramatic slow down in my code when I use a 
conversion like this (when looped over 10 million iterations 
for benchmarking).


Cheers!


Converting numbers to string involves the most expensive known 
two operations : division and modulus by 10.


When the base is known in advance, division and modulus can be 
substituted by a few additions, subtractions and bit shifts.  For 
example, the Hacker's Delight book has a chapter dedicated to 
that, as well as a freely available additional chapter 
(www.hackersdelight.org/divcMore.pdf).


Does DMD, or Phobos function to!(string), do anything like that?  
The number of possible bases is not large anyway.  I've heard 
major C/C++ compilers do that, but have not looked for a proof 
myself.

Re: Most performant way of converting int to string

[Daniel Kozák via Digitalmars-d-learn]

V Tue, 22 Dec 2015 09:43:00 -0800
"H. S. Teoh via Digitalmars-d-learn"
 napsáno:

> On Tue, Dec 22, 2015 at 05:23:11PM +, Andrew Chapman via
> Digitalmars-d-learn wrote: [...]
> > for({int i; i = 0;} i < num; i++) {
> > //string s = to!string(i);
> > Customer c = Customer(i, "Customer", "", i
> > * 2); string result = objS.serialize(c);
> > }
> > 
> > If I uncomment the "string s" line I'm seeing a 20% increase in
> > running time, which given what's going on the rest of the code is
> > quite surprising.  I've tried compiling with both dmd and ldc2 -
> > it's the same under both.  
> [...]
> 
> I wonder if the slowdown is caused by GC collection cycles (because
> calling to!string will allocate, and here you're making a very large
> number of small allocations, which is known to cause GC performance
> issues).
> 
> Try inserting this before the loop:
> 
>   import core.memory;
>   GC.disable();
> 
> Does this make a difference in the running time?
> 
> 
> T
> 
This would  not help. It would probably be worse.

Re: Most performant way of converting int to string

[Andrew Chapman via Digitalmars-d-learn]

On Tuesday, 22 December 2015 at 17:43:00 UTC, H. S. Teoh wrote:

I wonder if the slowdown is caused by GC collection cycles 
(because calling to!string will allocate, and here you're 
making a very large number of small allocations, which is known 
to cause GC performance issues).


Try inserting this before the loop:

import core.memory;
GC.disable();

Does this make a difference in the running time?


T


Thanks!  Unfortunately that actually makes it run slightly slower 
:-)

Re: Most performant way of converting int to string

[Daniel Kozák via Digitalmars-d-learn]

V Tue, 22 Dec 2015 12:55:10 -0800
"H. S. Teoh via Digitalmars-d-learn"
 napsáno:

> On Tue, Dec 22, 2015 at 08:54:35PM +0100, Daniel Kozák via
> Digitalmars-d-learn wrote:
> > V Tue, 22 Dec 2015 09:43:00 -0800
> > "H. S. Teoh via Digitalmars-d-learn"
> >  napsáno:
> >   
> > > On Tue, Dec 22, 2015 at 05:23:11PM +, Andrew Chapman via
> > > Digitalmars-d-learn wrote: [...]  
> > > > for({int i; i = 0;} i < num; i++) {
> > > > //string s = to!string(i);
> > > > Customer c = Customer(i, "Customer",
> > > > "", i
> > > > * 2); string result = objS.serialize(c);
> > > > }
> > > > 
> > > > If I uncomment the "string s" line I'm seeing a 20% increase in
> > > > running time, which given what's going on the rest of the code
> > > > is quite surprising.  I've tried compiling with both dmd and
> > > > ldc2 - it's the same under both.
> > > [...]
> > > 
> > > I wonder if the slowdown is caused by GC collection cycles
> > > (because calling to!string will allocate, and here you're making
> > > a very large number of small allocations, which is known to cause
> > > GC performance issues).
> > > 
> > > Try inserting this before the loop:
> > > 
> > >   import core.memory;
> > >   GC.disable();
> > > 
> > > Does this make a difference in the running time?
> > > 
> > > 
> > > T
> > >   
> > This would  not help. It would probably be worse.  
> 
> I was not suggesting this as a solution, it's merely a way to
> determine whether the performance issue is GC-related.
> 
> 
> T
> 

Ok :)

Re: Most performant way of converting int to string

[cym13 via Digitalmars-d-learn]

On Tuesday, 22 December 2015 at 17:15:27 UTC, Andrew Chapman 
wrote:
Sorry if this is a silly question but is the to! method from 
the conv library the most efficient way of converting an 
integer value to a string?


e.g.
string s = to!string(100);

I'm seeing a pretty dramatic slow down in my code when I use a 
conversion like this (when looped over 10 million iterations 
for benchmarking).


Cheers!


Out of curiosity a slow down compared to what? No conversion at 
all?

Re: Most performant way of converting int to string

[H. S. Teoh via Digitalmars-d-learn]

On Tue, Dec 22, 2015 at 05:23:11PM +, Andrew Chapman via 
Digitalmars-d-learn wrote:
[...]
> for({int i; i = 0;} i < num; i++) {
> //string s = to!string(i);
> Customer c = Customer(i, "Customer", "", i * 2);
> string result = objS.serialize(c);
> }
> 
> If I uncomment the "string s" line I'm seeing a 20% increase in
> running time, which given what's going on the rest of the code is
> quite surprising.  I've tried compiling with both dmd and ldc2 - it's
> the same under both.
[...]

I wonder if the slowdown is caused by GC collection cycles (because
calling to!string will allocate, and here you're making a very large
number of small allocations, which is known to cause GC performance
issues).

Try inserting this before the loop:

import core.memory;
GC.disable();

Does this make a difference in the running time?


T

-- 
"Uhh, I'm still not here." -- KD, while "away" on ICQ.

Re: Most performant way of converting int to string

[Andrew Chapman via Digitalmars-d-learn]

On Tuesday, 22 December 2015 at 17:18:16 UTC, cym13 wrote:
On Tuesday, 22 December 2015 at 17:15:27 UTC, Andrew Chapman 
wrote:
Sorry if this is a silly question but is the to! method from 
the conv library the most efficient way of converting an 
integer value to a string?


e.g.
string s = to!string(100);

I'm seeing a pretty dramatic slow down in my code when I use a 
conversion like this (when looped over 10 million iterations 
for benchmarking).


Cheers!


Out of curiosity a slow down compared to what? No conversion at 
all?


Yeah, if I include a simple conversion in my loop:

for({int i; i = 0;} i < num; i++) {
//string s = to!string(i);
Customer c = Customer(i, "Customer", "", 
i * 2);

string result = objS.serialize(c);
}

If I uncomment the "string s" line I'm seeing a 20% increase in 
running time, which given what's going on the rest of the code is 
quite surprising.  I've tried compiling with both dmd and ldc2 - 
it's the same under both.


Cheers.

Re: Most performant way of converting int to string

[cym13 via Digitalmars-d-learn]

On Tuesday, 22 December 2015 at 17:52:52 UTC, Andrew Chapman 
wrote:

On Tuesday, 22 December 2015 at 17:43:00 UTC, H. S. Teoh wrote:

I wonder if the slowdown is caused by GC collection cycles 
(because calling to!string will allocate, and here you're 
making a very large number of small allocations, which is 
known to cause GC performance issues).


Try inserting this before the loop:

import core.memory;
GC.disable();

Does this make a difference in the running time?


T


Thanks!  Unfortunately that actually makes it run slightly 
slower :-)


I dislike guessing so here are my thought on it and the method I 
used to get there:


First I used the following code:

void f() {
import std.conv;

string s;
foreach (i ; 0..1_000_000) {
s = i.to!string;
}
}

void g() {
import core.memory;
import std.conv;

GC.disable;

string s;
foreach (i ; 0..1_000_000) {
s = i.to!string;
}
}

extern (C) int snprintf(char*, size_t, const char*, ...);
void h() {
char[10] s;
foreach (i ; 0..1_000_000) {
snprintf(cast(char*)s, 10, cast(char*)"%d", i);
}
}

void main(string[] args) {
   f;
// g;
// h;
}

Note that for h I didn't really use a string, I used a char[10]. 
We should
keep in mind that you may want to convert it to a proper string 
later.


I compiled it three times (one per function) with `dmd -profile 
-profile=gc`
and then ran it to get profiling data. I then compiled it with 
`dmd` while
timing each execution 6 times and discarding the first time (as 
the first one
has to heat the cache). The reason for the recompilation is that 
as the C
code can't be profiled and profiling puts some overhead I wanted 
to get that

off my shoulders while timing.

Here are the average times:

f: 0.50 user 0.00 system 95% CPU 0.530 total

g: 0.49 user 0.01 system 96% CPU 0.522 total

h: 0.17 user 0.00 system 92% CPU 0.188 total

The GC profiler didn't find any allocation be it in f, g or h.

The profiler was only available for f and g with very similar 
results (as

they are the same function really, here is the interesting part):

 Timer Is 3579545 Ticks/Sec, Times are in Microsecs 



  Num  TreeFuncPer
  CallsTimeTimeCall

100   390439368   265416744 265 pure nothrow 
@safe char[] std.array.array!(std.conv.toChars!(10, char, 1, 
int).toChars(int).Result).array(std.conv.toChars!(10, char, 1, 
int).toChars(int).Result)
1008322499483224994  83 pure nothrow ref 
@nogc @safe std.conv.toChars!(10, char, 1, 
int).toChars(int).Result std.conv.toChars!(10, char, 1, 
int).toChars(int).Result.__ctor(int)

2370 118276850773190160   3 _Dmain
100   52573232835191373  35 pure nothrow 
@trusted immutable(char)[] std.conv.toImpl!(immutable(char)[], 
int).toImpl(int, uint, std.ascii.LetterCase)
5904369565933204064   5 pure nothrow ref 
@nogc @safe char std.conv.emplaceRef!(char, char).emplaceRef(ref 
char, ref char)
1003274590932745909  32 pure nothrow 
char[] std.array.arrayAllocImpl!(false, char[], 
ulong).arrayAllocImpl(ulong)
100   10010158616876591  16 pure nothrow 
@nogc @safe std.conv.toChars!(10, char, 1, 
int).toChars(int).Result std.conv.toChars!(10, char, 1, 
int).toChars(int)
5901307001413070014   2 pure nothrow 
@property @nogc @safe char std.conv.toChars!(10, char, 1, 
int).toChars(int).Result.front()
1004489944412153535  12 pure nothrow 
@trusted char[] std.array.uninitializedArray!(char[], 
ulong).uninitializedArray(ulong)
5901049159510491595   1 pure nothrow ref 
@nogc @safe char 
std.conv.emplaceImpl!(char).emplaceImpl!(char).emplaceImpl(ref 
char, ref char)
690 9952971 9952971   1 pure nothrow 
@property @nogc @safe bool std.conv.toChars!(10, char, 1, 
int).toChars(int).Result.empty()
10053086148 8186704   8 pure nothrow 
@trusted char[] std.array.array!(std.conv.toChars!(10, char, 1, 
int).toChars(int).Result).array(std.conv.toChars!(10, char, 1, 
int).toChars(int).Result).__lambda2()
100   530599849 4867521   4 pure nothrow 
@safe immutable(char)[] std.conv.toImpl!(immutable(char)[], 
int).toImpl(int)
100   534801618 4201769   4 pure nothrow 
@safe immutable(char)[] 
std.conv.to!(immutable(char)[]).to!(int).to(int)
590 2520677 2520677   0 pure nothrow 
@nogc @safe void std.conv.toChars!(10, char, 1, 
int).toChars(int).Result.popFront()
100 2138919 2138919   2 pure nothrow 
@nogc @trusted char[] std.array.array!(std.conv.toChars!(10, 
char, 1, int).toChars(int).Result).array(std.conv.toChars!(10, 

Re: Most performant way of converting int to string

[rumbu via Digitalmars-d-learn]

On Tuesday, 22 December 2015 at 19:45:46 UTC, Daniel Kozák wrote:

V Tue, 22 Dec 2015 18:11:24 +
rumbu via Digitalmars-d-learn 


napsáno:

On Tuesday, 22 December 2015 at 17:15:27 UTC, Andrew Chapman 
wrote:
> Sorry if this is a silly question but is the to! method from 
> the conv library the most efficient way of converting an 
> integer value to a string?

>
> e.g.
> string s = to!string(100);
>
> I'm seeing a pretty dramatic slow down in my code when I use 
> a conversion like this (when looped over 10 million 
> iterations for benchmarking).

>
> Cheers!

Converting numbers to string involves the most expensive known 
two operations : division and modulus by 10.


No, IIRC few months or maybe years I have optimize this so it 
does not use division and modulus


It's using division and modulus, as expected:

https://github.com/D-Programming-Language/phobos/blob/master/std/conv.d#L5529

If the compiler is smart enough, maybe it will replace this by 
well known multiplication trick.