Re: CompileTime performance measurement

[Marco Leise via Digitalmars-d]

Am Tue, 06 Sep 2016 05:02:54 +
schrieb timepp :

> On Sunday, 4 September 2016 at 04:24:34 UTC, rikki cattermole 
> wrote:
> > void writeln(T...)(T args) {
> > if (__ctfe){
> > debug {
> > __ctfeWriteln(args);
> > }
> > } else {
> > // ... current implementation
> > }
> > }
> >
> > Are you sure?  
> any usage example?
> 
> consider a normal usage:
> writeln("done.");
> 
> I just want a runtime output. how can I tell the compiler not to 
> print "done." at compile time?

If you actually call a function during compile-time that uses
writeln and want it to be silent you would write:

  if (!__ctfe) writeln("done.");

-- 
Marco

Re: CompileTime performance measurement

[Stefan Koch via Digitalmars-d]

On Thursday, 8 September 2016 at 17:15:54 UTC, safety0ff wrote:
On Thursday, 8 September 2016 at 17:03:30 UTC, Stefan Koch 
wrote:


I thought of the same thing a while back.
However I have had the time to decipher the gprof data-format 
yet.
Is there another profile-format for decent visualization tools 
exist ?


I was just using that as an example of what we might want to 
output as text.
e.g. 
https://sourceware.org/binutils/docs/gprof/Flat-Profile.html


I wasn't saying that we should mimic gmon.out file format, I 
don't think that buys us much.


I disagree anything which allows to use existing visualization 
and correlation will be a major win.

If I am going to write a profiler it should have pretty charts.
Also the gnu guys probably put a lot thought into their format.

Re: CompileTime performance measurement

[safety0ff via Digitalmars-d]

On Thursday, 8 September 2016 at 17:03:30 UTC, Stefan Koch wrote:


I thought of the same thing a while back.
However I have had the time to decipher the gprof data-format 
yet.
Is there another profile-format for decent visualization tools 
exist ?


I was just using that as an example of what we might want to 
output as text.

e.g. https://sourceware.org/binutils/docs/gprof/Flat-Profile.html

I wasn't saying that we should mimic gmon.out file format, I 
don't think that buys us much.

Re: CompileTime performance measurement

[Stefan Koch via Digitalmars-d]

On Thursday, 8 September 2016 at 16:52:47 UTC, safety0ff wrote:

On Sunday, 4 September 2016 at 00:04:16 UTC, Stefan Koch wrote:


... I have now implemented another pseudo function called 
__ctfeTicksMs.

[Snip]

This does allow meaningful compiletime performance tests to be 
written.

spanning both CTFE and template-incitations timeings.

Please tell me what you think.


I think automated ctfe profiling would be much better and the 
byte-code interpreter seems like a great platform to build this 
onto.


For example, using a command line switch to enable profiling 
which outputs something similar to gprof's flat profile.


I thought of the same thing a while back.
However I have had the time to decipher the gprof data-format yet.
Is there another profile-format for decent visualization tools 
exist ?

Re: CompileTime performance measurement

[safety0ff via Digitalmars-d]

On Sunday, 4 September 2016 at 00:04:16 UTC, Stefan Koch wrote:


... I have now implemented another pseudo function called 
__ctfeTicksMs.

[Snip]

This does allow meaningful compiletime performance tests to be 
written.

spanning both CTFE and template-incitations timeings.

Please tell me what you think.


I think automated ctfe profiling would be much better and the 
byte-code interpreter seems like a great platform to build this 
onto.


For example, using a command line switch to enable profiling 
which outputs something similar to gprof's flat profile.


Skimming the byte-code work it seems like it is too early to add 
this yet.


My thoughts on __ctfeTicksMs:
- it isn't very meaningful for users without intimate compiler 
knowledge

- it requires writing boilerplate code over and over for profiling
- doesn't seem like it would work well for functions that get 
executed multiple times


While it might be a useful compiler developer hack, I do not 
think it should become a user primitive.

Re: CompileTime performance measurement

[Stefan Koch via Digitalmars-d]

On Wednesday, 7 September 2016 at 06:49:09 UTC, Rory McGuire 
wrote:


Seriously Stefan, you make my day!

My libraries will be so much easier to write!


I am glad the time was not wasted.
Let's hope it gets merged :)

Re: CompileTime performance measurement

[Rory McGuire via Digitalmars-d]

On Tue, Sep 6, 2016 at 7:42 PM, Stefan Koch via Digitalmars-d <
digitalmars-d@puremagic.com> wrote:

> On Tuesday, 6 September 2016 at 10:42:00 UTC, Martin Nowak wrote:
>
>> On Sunday, 4 September 2016 at 00:04:16 UTC, Stefan Koch wrote:
>>
>>> I recently implemented __ctfeWriteln.
>>>
>>
>> Nice, is it only for your interpreter or can we move
>> https://trello.com/c/6nU0lbl2/24-ctfewrite to done? I think __ctfeWrite
>> would be a better primitive. And we could actually consider to specialize
>> std.stdio.write* for CTFE.
>>
>
> It's only for the current engine and only for Strings!
> See: https://github.com/dlang/druntime/pull/1643
> and https://github.com/dlang/dmd/pull/6101
>



Seriously Stefan, you make my day!

My libraries will be so much easier to write!

Re: CompileTime performance measurement

[Stefan Koch via Digitalmars-d]

On Tuesday, 6 September 2016 at 10:42:00 UTC, Martin Nowak wrote:

On Sunday, 4 September 2016 at 00:04:16 UTC, Stefan Koch wrote:

I recently implemented __ctfeWriteln.


Nice, is it only for your interpreter or can we move 
https://trello.com/c/6nU0lbl2/24-ctfewrite to done? I think 
__ctfeWrite would be a better primitive. And we could actually 
consider to specialize std.stdio.write* for CTFE.


It's only for the current engine and only for Strings!
See: https://github.com/dlang/druntime/pull/1643
and https://github.com/dlang/dmd/pull/6101

Re: CompileTime performance measurement

[Jonathan M Davis via Digitalmars-d]

On Tuesday, September 06, 2016 10:46:11 Martin Nowak via Digitalmars-d wrote:
> On Sunday, 4 September 2016 at 00:04:16 UTC, Stefan Koch wrote:
> > Hi Guys.
> >
> > I recently implemented __ctfeWriteln.
> > Based on that experience I have now implemented another pseudo
> > function called __ctfeTicksMs.
> > That evaluates to a uint representing the number of
> > milliseconds elapsed between the start of dmd and the time of
> > semantic evaluation of this expression.
>
> For bigger CTFE programs it might be helpful.
> Milliseconds are a fairly low resolution, would think with hnsec
> or so makes a better unit. Using core.time.TickDuration for that
> would make sense.

If you're doing to do that use core.time.Duration. TickDuration is slated to
be deprecated once the functionality in Phobos that uses it has been
deprecated. Duration replaces its functionality as a duration, and MonoTime
replaces its functionality as a timestamp of the monotonic clock.

- Jonathan M Davis

Re: CompileTime performance measurement

[Martin Nowak via Digitalmars-d]

On Sunday, 4 September 2016 at 00:04:16 UTC, Stefan Koch wrote:

Hi Guys.

I recently implemented __ctfeWriteln.
Based on that experience I have now implemented another pseudo 
function called __ctfeTicksMs.
That evaluates to a uint representing the number of 
milliseconds elapsed between the start of dmd and the time of 
semantic evaluation of this expression.


For bigger CTFE programs it might be helpful.
Milliseconds are a fairly low resolution, would think with hnsec 
or so makes a better unit. Using core.time.TickDuration for that 
would make sense.

Re: CompileTime performance measurement

[Martin Nowak via Digitalmars-d]

On Sunday, 4 September 2016 at 00:04:16 UTC, Stefan Koch wrote:

I recently implemented __ctfeWriteln.


Nice, is it only for your interpreter or can we move 
https://trello.com/c/6nU0lbl2/24-ctfewrite to done? I think 
__ctfeWrite would be a better primitive. And we could actually 
consider to specialize std.stdio.write* for CTFE.

Re: CompileTime performance measurement

[Martin Nowak via Digitalmars-d]

On Sunday, 4 September 2016 at 00:08:14 UTC, David Nadlinger 
wrote:

Please don't. This makes CTFE indeterministic.


Well we already have __TIMESTAMP__, though I think it doesn't 
change during compilation.

Re: CompileTime performance measurement

[Martin Tschierschke via Digitalmars-d]

On Sunday, 4 September 2016 at 19:36:16 UTC, Stefan Koch wrote:
On Sunday, 4 September 2016 at 12:38:05 UTC, Andrei 
Alexandrescu wrote:

On 9/4/16 6:14 AM, Stefan Koch wrote:
writeln and __ctfeWriteln are to be regarded as completely 
different

things.
__ctfeWriteln is a debugging tool only!
It should not be used in any production code.


Well I'm not sure how that would be reasonably enforced. -- 
Andrei


One could enforce it by defining it inside a version or debug 
block.
The reason I do not want to see this in production code is as 
follows:


In the engine I am working on, communication between it and the 
rest of dmd is kept to a minimum, because :


"The new CTFE engine abstracts away everything into bytecode,
there is no guarantee that the bytecode-evaluator is run in the 
same process or even on the same machine."


An alternative might be, to save your ctfe values in an static 
array and output them on startup of the compiled program. Same 
idea is used in vibe.d to make a caching of the templates 
evaluation possible. See: http://code.dlang.org/packages/diet-ng 
Experimental HTML template caching

Re: CompileTime performance measurement

[timepp via Digitalmars-d]

On Sunday, 4 September 2016 at 04:24:34 UTC, rikki cattermole 
wrote:

void writeln(T...)(T args) {
if (__ctfe){
debug {
__ctfeWriteln(args);
}
} else {
// ... current implementation
}
}

Are you sure?

any usage example?

consider a normal usage:
writeln("done.");

I just want a runtime output. how can I tell the compiler not to 
print "done." at compile time?

Re: CompileTime performance measurement

[Stefan Koch via Digitalmars-d]

On Sunday, 4 September 2016 at 12:38:05 UTC, Andrei Alexandrescu 
wrote:

On 9/4/16 6:14 AM, Stefan Koch wrote:
writeln and __ctfeWriteln are to be regarded as completely 
different

things.
__ctfeWriteln is a debugging tool only!
It should not be used in any production code.


Well I'm not sure how that would be reasonably enforced. -- 
Andrei


One could enforce it by defining it inside a version or debug 
block.
The reason I do not want to see this in production code is as 
follows:


In the engine I am working on, communication between it and the 
rest of dmd is kept to a minimum, because :


"The new CTFE engine abstracts away everything into bytecode,
there is no guarantee that the bytecode-evaluator is run in the 
same process or even on the same machine."

Re: CompileTime performance measurement

[Andrei Alexandrescu via Digitalmars-d]

On 9/4/16 6:14 AM, Stefan Koch wrote:

writeln and __ctfeWriteln are to be regarded as completely different
things.
__ctfeWriteln is a debugging tool only!
It should not be used in any production code.


Well I'm not sure how that would be reasonably enforced. -- Andrei

Re: CompileTime performance measurement

[Andrei Alexandrescu via Digitalmars-d]

On 9/4/16 6:10 AM, rikki cattermole wrote:

On 04/09/2016 2:08 PM, Stefan Koch wrote:

On Sunday, 4 September 2016 at 02:06:55 UTC, Stefan Koch wrote:

This works already.
Anything placed in a debug {} block will be considered pure regardless.


Opps your comment was about the debate.
I would say that __ctfeWriteln and __ctfeTicksMs should not work outside
of debug.


Can we have writeln and writefln call into it if __ctfe is true?
Just so that we have got some consistency between runtime and CTFE usage.


Yes, that's the natural way. __ctfeWriteln is really removing a 
limitation of writeln. -- Andrei

Re: CompileTime performance measurement

[Stefan Koch via Digitalmars-d]

On Sunday, 4 September 2016 at 04:31:09 UTC, Jonathan M Davis 
wrote:


He didn't say that it _couldn't_ be done. He said that it 
_shouldn't_ be done.


- Jonathan M Davis


Yes exactly.

Re: CompileTime performance measurement

[Stefan Koch via Digitalmars-d]

On Sunday, 4 September 2016 at 04:35:15 UTC, rikki cattermole 
wrote:

void writeln(T...)(T args) {
  if (__ctfe){
  debug {
  __ctfeWriteln(args);
  }
  } else {
  // ... current implementation
  }
}


That will not work.
The signature is void __ctfeWriteln(const string s)

Re: CompileTime performance measurement

[rikki cattermole via Digitalmars-d]

On 04/09/2016 4:31 PM, Jonathan M Davis via Digitalmars-d wrote:

On Sunday, September 04, 2016 16:24:34 rikki cattermole via Digitalmars-d
wrote:

On 04/09/2016 4:14 PM, Stefan Koch wrote:

On Sunday, 4 September 2016 at 04:10:29 UTC, rikki cattermole wrote:

On 04/09/2016 2:08 PM, Stefan Koch wrote:

On Sunday, 4 September 2016 at 02:06:55 UTC, Stefan Koch wrote:

This works already.
Anything placed in a debug {} block will be considered pure regardless.


Opps your comment was about the debate.
I would say that __ctfeWriteln and __ctfeTicksMs should not work outside
of debug.


Can we have writeln and writefln call into it if __ctfe is true?
Just so that we have got some consistency between runtime and CTFE usage.


No!
writeln and __ctfeWriteln are to be regarded as completely different
things.
__ctfeWriteln is a debugging tool only!
It should not be used in any production code.


void writeln(T...)(T args) {
  if (__ctfe){
  debug {
  __ctfeWriteln(args);
  }
  } else {
  // ... current implementation
  }
}

Are you sure?


He didn't say that it _couldn't_ be done. He said that it _shouldn't_ be
done.

- Jonathan M Davis


I wrote that code to prove that it could be used only for debugging 
purposes, which is Stefan's argument.

Re: CompileTime performance measurement

[Jonathan M Davis via Digitalmars-d]

On Sunday, September 04, 2016 16:24:34 rikki cattermole via Digitalmars-d 
wrote:
> On 04/09/2016 4:14 PM, Stefan Koch wrote:
> > On Sunday, 4 September 2016 at 04:10:29 UTC, rikki cattermole wrote:
> >> On 04/09/2016 2:08 PM, Stefan Koch wrote:
> >>> On Sunday, 4 September 2016 at 02:06:55 UTC, Stefan Koch wrote:
>  This works already.
>  Anything placed in a debug {} block will be considered pure regardless.
> >>>
> >>> Opps your comment was about the debate.
> >>> I would say that __ctfeWriteln and __ctfeTicksMs should not work outside
> >>> of debug.
> >>
> >> Can we have writeln and writefln call into it if __ctfe is true?
> >> Just so that we have got some consistency between runtime and CTFE usage.
> >
> > No!
> > writeln and __ctfeWriteln are to be regarded as completely different
> > things.
> > __ctfeWriteln is a debugging tool only!
> > It should not be used in any production code.
>
> void writeln(T...)(T args) {
>   if (__ctfe){
>   debug {
>   __ctfeWriteln(args);
>   }
>   } else {
>   // ... current implementation
>   }
> }
>
> Are you sure?

He didn't say that it _couldn't_ be done. He said that it _shouldn't_ be
done.

- Jonathan M Davis

Re: CompileTime performance measurement

[rikki cattermole via Digitalmars-d]

On 04/09/2016 4:14 PM, Stefan Koch wrote:

On Sunday, 4 September 2016 at 04:10:29 UTC, rikki cattermole wrote:

On 04/09/2016 2:08 PM, Stefan Koch wrote:

On Sunday, 4 September 2016 at 02:06:55 UTC, Stefan Koch wrote:

This works already.
Anything placed in a debug {} block will be considered pure regardless.


Opps your comment was about the debate.
I would say that __ctfeWriteln and __ctfeTicksMs should not work outside
of debug.


Can we have writeln and writefln call into it if __ctfe is true?
Just so that we have got some consistency between runtime and CTFE usage.


No!
writeln and __ctfeWriteln are to be regarded as completely different
things.
__ctfeWriteln is a debugging tool only!
It should not be used in any production code.


void writeln(T...)(T args) {
if (__ctfe){
debug {
__ctfeWriteln(args);
}
} else {
// ... current implementation
}
}

Are you sure?

Re: CompileTime performance measurement

[Stefan Koch via Digitalmars-d]

On Sunday, 4 September 2016 at 04:10:29 UTC, rikki cattermole 
wrote:

On 04/09/2016 2:08 PM, Stefan Koch wrote:

On Sunday, 4 September 2016 at 02:06:55 UTC, Stefan Koch wrote:

This works already.
Anything placed in a debug {} block will be considered pure 
regardless.


Opps your comment was about the debate.
I would say that __ctfeWriteln and __ctfeTicksMs should not 
work outside

of debug.


Can we have writeln and writefln call into it if __ctfe is true?
Just so that we have got some consistency between runtime and 
CTFE usage.


No!
writeln and __ctfeWriteln are to be regarded as completely 
different things.

__ctfeWriteln is a debugging tool only!
It should not be used in any production code.

Re: CompileTime performance measurement

[rikki cattermole via Digitalmars-d]

On 04/09/2016 2:08 PM, Stefan Koch wrote:

On Sunday, 4 September 2016 at 02:06:55 UTC, Stefan Koch wrote:

This works already.
Anything placed in a debug {} block will be considered pure regardless.


Opps your comment was about the debate.
I would say that __ctfeWriteln and __ctfeTicksMs should not work outside
of debug.


Can we have writeln and writefln call into it if __ctfe is true?
Just so that we have got some consistency between runtime and CTFE usage.

Re: CompileTime performance measurement

[Stefan Koch via Digitalmars-d]

On Sunday, 4 September 2016 at 02:06:55 UTC, Stefan Koch wrote:

This works already.
Anything placed in a debug {} block will be considered pure 
regardless.


Opps your comment was about the debate.
I would say that __ctfeWriteln and __ctfeTicksMs should not work 
outside of debug.

Re: CompileTime performance measurement

[Stefan Koch via Digitalmars-d]

On Sunday, 4 September 2016 at 02:03:49 UTC, sarn wrote:

On Sunday, 4 September 2016 at 01:53:21 UTC, Stefan Koch wrote:

Pragma msg can only print compiletime constants.
While __ctfeWriteln can print state while doing CTFE.


Thanks, that makes a lot of sense.  Just to check, it prints to 
standard error, right?


Also, the issue of non-deterministic compilation reminds me of 
the debate about allowing logging statements in pure functions.
 Maybe there's a similar answer (i.e., making it only work in 
some kind of debug mode).


This works already.
Anything placed in a debug {} block will be considered pure 
regardless.

Re: CompileTime performance measurement

[sarn via Digitalmars-d]

On Sunday, 4 September 2016 at 01:53:21 UTC, Stefan Koch wrote:

Pragma msg can only print compiletime constants.
While __ctfeWriteln can print state while doing CTFE.


Thanks, that makes a lot of sense.  Just to check, it prints to 
standard error, right?


Also, the issue of non-deterministic compilation reminds me of 
the debate about allowing logging statements in pure functions.  
Maybe there's a similar answer (i.e., making it only work in some 
kind of debug mode).

Re: CompileTime performance measurement

[Stefan Koch via Digitalmars-d]

On Sunday, 4 September 2016 at 01:44:40 UTC, sarn wrote:

On Sunday, 4 September 2016 at 00:04:16 UTC, Stefan Koch wrote:

I recently implemented __ctfeWriteln.


Sounds like pragma msg.  How does it compare?
https://dlang.org/spec/pragma.html#msg


Pragma msg can only print compiletime constants.
While __ctfeWriteln can print state while doing CTFE.

Example
int fn(int n)
{
  import std.conv;
  __ctfeWriteln((n-10).to!string);
  return n;
}

static assert(fn(22));

will print 12;
whereas
int fn(int n)
{
  import std.conv;
  pragma(msg, n.to!string);
  return n;
}
will tell you that the symbol n is not avilable at compiletime

Re: CompileTime performance measurement

[sarn via Digitalmars-d]

On Sunday, 4 September 2016 at 00:04:16 UTC, Stefan Koch wrote:

I recently implemented __ctfeWriteln.


Sounds like pragma msg.  How does it compare?
https://dlang.org/spec/pragma.html#msg

Re: CompileTime performance measurement

[Stefan Koch via Digitalmars-d]

On Sunday, 4 September 2016 at 00:08:14 UTC, David Nadlinger 
wrote:


Please don't. This makes CTFE indeterministic.


Please elaborate on why this would have a negative impact ?
if someone chooses to use a symbol called __ctfeTicksMs they 
shoud know what they are doing.


To write performance tests, just measure compilation of a whole 
program (possibly with -o-). The variance due to the 
startup/shutdown overhead can trivially be controlled by just 
executing the CTFE code in question often enough.


That will only allow you to tell how much overall ctfe-time you 
spent.

It will not allow you to pinpoint and optimize the offenders.

Re: CompileTime performance measurement

[David Nadlinger via Digitalmars-d]

On Sunday, 4 September 2016 at 00:04:16 UTC, Stefan Koch wrote:
Based on that experience I have now implemented another pseudo 
function called __ctfeTicksMs.

[…]
Please tell me what you think.


Please don't. This makes CTFE indeterministic.

To write performance tests, just measure compilation of a whole 
program (possibly with -o-). The variance due to the 
startup/shutdown overhead can trivially be controlled by just 
executing the CTFE code in question often enough.


 — David

CompileTime performance measurement

[Stefan Koch via Digitalmars-d]

Hi Guys.

I recently implemented __ctfeWriteln.
Based on that experience I have now implemented another pseudo 
function called __ctfeTicksMs.
That evaluates to a uint representing the number of milliseconds 
elapsed between the start of dmd and the time of semantic 
evaluation of this expression.


This does allow meaningful compiletime performance tests to be 
written.

spanning both CTFE and template-incitations timeings.

Please tell me what you think.