Re: CTFE Status 2
On 7/12/17 9:20 PM, H. S. Teoh via Digitalmars-d wrote: On Thu, Jul 13, 2017 at 01:23:53AM +, Stefan Koch via Digitalmars-d wrote: On Wednesday, 12 July 2017 at 21:05:28 UTC, watcher wrote: stop advertising yourself and polluting this thread. seek help - no down-payments in Germany!! I am not sure what you mean ... I assume it's a joke ? Or a troll. Just ignore it. I kind of like Stefan's response though :) Carry on, Stefan! Though here is a suggestion -- start a new thread every month. Every time you post a response on this thread that's not in response to something, it jumps the entire history back up to the top of the list. -Steve
Re: CTFE Status 2
On Thu, Jul 13, 2017 at 01:23:53AM +, Stefan Koch via Digitalmars-d wrote: > On Wednesday, 12 July 2017 at 21:05:28 UTC, watcher wrote: > > > > stop advertising yourself and polluting this thread. > > seek help - no down-payments in Germany!! > > I am not sure what you mean ... > I assume it's a joke ? Or a troll. Just ignore it. T -- Debian GNU/Linux: Cray on your desktop.
Re: CTFE Status 2
On Wednesday, 12 July 2017 at 21:05:28 UTC, watcher wrote: stop advertising yourself and polluting this thread. seek help - no down-payments in Germany!! I am not sure what you mean ... I assume it's a joke ? If you are indeed offering help, you can reproduce this by checking out https://github.com/UplinkCoder/dmd/tree/newCTFE_on_master and compiling tests/runnable/template8.d
Re: CTFE Status 2
On Wednesday, 12 July 2017 at 18:57:12 UTC, Stefan Koch wrote: On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote: [ ... ] Another ABI issue just pooped on my Wednesday. this time it manifests while doing concatenation inside a template the test failing is (test/runnable/template8.d) This is again hard to debug because I have to piece together the actual source-code which is being executed. templates are a pain in the butt! stop advertising yourself and polluting this thread. seek help - no down-payments in Germany!!
Re: CTFE Status 2
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote: [ ... ] Another ABI issue just pooped on my Wednesday. this time it manifests while doing concatenation inside a template the test failing is (test/runnable/template8.d) This is again hard to debug because I have to piece together the actual source-code which is being executed. templates are a pain in the butt!
Re: CTFE Status 2
On Wednesday, 12 July 2017 at 09:57:21 UTC, Martin Tschierschke wrote: On Wednesday, 12 July 2017 at 07:58:30 UTC, Stefan Koch wrote: On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote: [ ... ] long story short: This is now fixed. and ABI bugs are HELL! Cheers, Stefan As being a newbie, could you please point to a post describing what you are doing in more general terms. Answering the question, what will the new CTFE bring to us all and when should we start preparing a big celebration for the time it gets part of DMD? Here is a blog-port describing the project. http://dlang.org/blog/2017/04/10/the-new-ctfe-engine/ newCTFE will reduce the time taken for ctfe by roughly 10x And, more importantly it reduces the memory-usage. Instead of a few gigabyte it will not even allocate more then a megabyte. (Currently it is hard capped to a maximum of 64K stack space and 16M heap).
Re: CTFE Status 2
On Wednesday, 12 July 2017 at 09:46:39 UTC, Kagamin wrote:
On Wednesday, 12 July 2017 at 07:58:30 UTC, Stefan Koch wrote:
Whereas A struct is always stored __packed__ and aligned to 4
byte boundaries
struct S // size: 20
{
ubyte m1; // offset: 0
ulong m2; // offset: 4
ubyte m3; // offset: 12
ubyte m4; // offset: 16
}
Do you plan to keep it like that? This won't work on non-x86
architectures that don't allow unaligned memory access.
For the time being, yes I plan to keep it.
It is 4 byte aligned which is plenty of alignment.
For architectures which do not allow unaligned access we would
split the ulong into hi and lo parts.
64bit arithmetic can generally be assumed to take a slower path
in newCTFE.
Re: CTFE Status 2
On Wednesday, 12 July 2017 at 07:58:30 UTC, Stefan Koch wrote: On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote: [ ... ] long story short: This is now fixed. and ABI bugs are HELL! Cheers, Stefan As being a newbie, could you please point to a post describing what you are doing in more general terms. Answering the question, what will the new CTFE bring to us all and when should we start preparing a big celebration for the time it gets part of DMD?
Re: CTFE Status 2
On Wednesday, 12 July 2017 at 07:58:30 UTC, Stefan Koch wrote:
Whereas A struct is always stored __packed__ and aligned to 4
byte boundaries
struct S // size: 20
{
ubyte m1; // offset: 0
ulong m2; // offset: 4
ubyte m3; // offset: 12
ubyte m4; // offset: 16
}
Do you plan to keep it like that? This won't work on non-x86
architectures that don't allow unaligned memory access.
Re: CTFE Status 2
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
[ ... ]
Hey Guys I just fixed an ABI issue concerning slices and structs.
This issue took 2.5 weeks to find and fix.
So let me describe what was going on.
As you may know newCTFE defines it's own ABI (since it is a
platform in it's own right)
Slices are generally represented as 16Byte SliceDescriptors
of the form
struct SliceDesc
{
uint Base; // T* really. but always 4 bytes regardless
uint Length;
uint Capcity; // unused for now.
uint ExtraFlags // unused will be important for array-casts
}
Whereas A struct is always stored __packed__ and aligned to 4
byte boundaries
struct S // size: 20
{
ubyte m1; // offset: 0
ulong m2; // offset: 4
ubyte m3; // offset: 12
ubyte m4; // offset: 16
}
Now Slices inside of structs are stored by injecting the
corresponding sliceDescriptor
Meaning:
AS { uint[] a; uint aLen; } => { uint[4] aDesc; uint aLen }
However the code that is repsonsible for converting newCTFE-ABI
values to DMD-expressions was still assuming this data-layout
AS { uint[] a; uint aLen; } => { (uint[4])* aDescPtr; uint aLen }
which caused it to do an extra derefence and and actually
interpreting the sliceBase as a descriptor pointer.
Which of course produced complete garbage.
The reason why this took me so long to fix is because I suspected
that the conversion was using the new convention and the
byte-code generator was at fault.
And in my attemptes to fix the bytecode generation I introduced
even more ABI bugs.
long story short:
This is now fixed.
and
ABI bugs are HELL!
Cheers,
Stefan
Re: CTFE Status 2
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote: [ ... ] Hi Guys, It's been a while since my last post. I have now prepared the interpreter to support float <=> double casts as well as float <=> long double <=> long (int*) <=> (float*) // reinterpret (long*) <=> (double*) //reinterpret However is is not yet hooked up to newCTFE. As I need to think about the interface. Either I can go with a Cast and BitCast function (which are prone to strange behavior, if they used with the lowered IR) Or Add the more explicit F32ToF64, F64To32, F32ToSI, SIToF32, UIToF32, F64ToSI, SIToF64, UIToF64, Which adds 8 calls to the IR-API and looks unsightly.
Re: CTFE Status 2
On Monday, 3 July 2017 at 06:15:44 UTC, Stefan Koch wrote:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch
wrote:
[ ... ]
This code works now as well ;)
int maxFloat()
{
float prev = 1;
foreach(int i; 0 .. int.max)
{
if (i == prev++)
return i;
}
return int.max;
}
static assert(maxFloat() == 16777216);
Re: CTFE Status 2
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
[ ... ]
Hi Guys,
I am very pleased indeed about the following code now running in
ctfe.
int maxFloat()
{
float prev = 0;
float fOne = 1;
foreach(int i; 1 .. int.max)
{
if (i == prev)
return i-1;
else
prev += fOne;
}
return int.max;
}
static assert(maxFloat() == 16777216);
Re: CTFE Status 2
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote: [ ... ] I ran in trouble with floating point. This will probably take a week. The debugger will take probably a month (without gui).
Re: CTFE Status 2
On 6/29/17 4:24 AM, Stefan Koch wrote: On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote: [ ... ] I just "discovered" a bug in my debugging system. Which caused me to debug the wrong function for about 5 hours. Since I really could not see why the codegen would produce such vastly different code. (Almost as if it were generating code for a different function :/ ) The takeaway: Better Debuggers only safe you time if they are actually showing you correct information. If your debugger is buggy then your are in a world of hurts. I had my share of pain for today. The same is true for compilers that are buggy. Debugging code is horrible when the compiled code isn't doing what the source says. I've seen my share of that as well... The good thing is however, that the floating point stuff is going smooth so far. The x87 can be persuaded to almost obey the ieee standard if you issue a write after every operation. The better option would be to just use sse2 all the way. But whatever. I'm looking very much forward to newCTFE. Keep up the good work! -Steve
Re: CTFE Status 2
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote: [ ... ] I just "discovered" a bug in my debugging system. Which caused me to debug the wrong function for about 5 hours. Since I really could not see why the codegen would produce such vastly different code. (Almost as if it were generating code for a different function :/ ) The takeaway: Better Debuggers only safe you time if they are actually showing you correct information. If your debugger is buggy then your are in a world of hurts. I had my share of pain for today. The good thing is however, that the floating point stuff is going smooth so far. The x87 can be persuaded to almost obey the ieee standard if you issue a write after every operation. The better option would be to just use sse2 all the way. But whatever.
Re: CTFE Status 2
On Wed, Jun 28, 2017 at 04:19:02PM +, Stefan Koch via Digitalmars-d wrote: > On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote: > > [ ... ] > > Hi There, > > I just wrote the first parts of the ctfe debugger. > Ctfe-Breakpoints are working :) > > It looks like we will be able to have greatly enhanced debugging > experience soon. > (okay soonish ... since there are still a few tricky details to work out.) Awesome! T -- There are four kinds of lies: lies, damn lies, and statistics.
Re: CTFE Status 2
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote: [ ... ] Hi There, I just wrote the first parts of the ctfe debugger. Ctfe-Breakpoints are working :) It looks like we will be able to have greatly enhanced debugging experience soon. (okay soonish ... since there are still a few tricky details to work out.)
Re: CTFE Status 2
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
[ ... ]
I finally came around and implemented 64bit as a hacky extension
to the interpreter.
ulong[5] testArrayAssignL()
{
typeof(return) arr = 12;
arr[3] = 4;
arr[0] = 1;
return arr;
}
static immutable arrL = testArrayAssignL();
pragma(msg, arrL); // outputs: [1LU, 12LU, 12LU, 4LU, 12LU]
This should naturally extend to arrays of floats or doubles as
well.
Re: CTFE Status 2
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote: [ ... ] I just figured out howto fix the defaultArg situation. We just have to check at or slightly before we emit the Call instruction, if we have enough arguments for the function. If we don't we pull the missing ones from the default arguments. This does have the problem that we can generate the same default once per call. However we would have the same problem if a user typed the default argument manually. Therefore I argue we should emit the code rather then demanding something clever from the backends. Clever is usually the opposite of smart :) (In those kinds of things)
Re: CTFE Status 2
On Tuesday, 20 June 2017 at 19:37:47 UTC, David Nadlinger wrote:
But how much of the std.math code are you actually executing
with newCTFE? What I meant is that if the std.math{,special}
implementations of the various mathematical functions work,
there shouldn't be any egregious issues. I'm not sure how much
of it is usually run during CTFE, though.
— David
I just checked.
I execute exactly 0% of it.
It mostly bails out on || and &&.
Re: CTFE Status 2
On Tuesday, 20 June 2017 at 19:01:06 UTC, Stefan Koch wrote:
On Tuesday, 20 June 2017 at 18:58:36 UTC, David Nadlinger wrote:
On Tuesday, 20 June 2017 at 17:35:28 UTC, Stefan Koch wrote:
Hit me with brittle numeric code please!
[…]
Unfortunately this also broke the phobos unitttests since now
more of is attempted to be evaluated.
Just making sure that the Phobos unit tests pass at compile
time (with 64 bit reals, if that is what you support) should
be a good start.
— David
Fixed.
But how much of the std.math code are you actually executing with
newCTFE? What I meant is that if the std.math{,special}
implementations of the various mathematical functions work, there
shouldn't be any egregious issues. I'm not sure how much of it is
usually run during CTFE, though.
— David
Re: CTFE Status 2
On Tuesday, 20 June 2017 at 17:35:28 UTC, Stefan Koch wrote: At the same time 64bit integer support was expanded. Such that we can now return long and ulong values. Unfortunately this also broke the phobos unitttests since now more of is attempted to be evaluated. The reason this broke was because we do not handle default arguments in calls correctly. I have added a bailout such that we will not try to call functions with default arguments. with that we should pass phobos uinttests again.
Re: CTFE Status 2
On Tuesday, 20 June 2017 at 18:58:36 UTC, David Nadlinger wrote: On Tuesday, 20 June 2017 at 17:35:28 UTC, Stefan Koch wrote: Hit me with brittle numeric code please! […] Unfortunately this also broke the phobos unitttests since now more of is attempted to be evaluated. Just making sure that the Phobos unit tests pass at compile time (with 64 bit reals, if that is what you support) should be a good start. — David Fixed.
Re: CTFE Status 2
On Tuesday, 20 June 2017 at 17:35:28 UTC, Stefan Koch wrote: Hit me with brittle numeric code please! […] Unfortunately this also broke the phobos unitttests since now more of is attempted to be evaluated. Just making sure that the Phobos unit tests pass at compile time (with 64 bit reals, if that is what you support) should be a good start. — David
Re: CTFE Status 2
On Tuesday, 20 June 2017 at 12:07:00 UTC, Stefan Koch wrote:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch
wrote:
[ ... ]
limited Support for 32bit floating point ops [+, -, *, /, %]
has just landed in newCTFE.
float fmadd(float a, float b, float c)
{
return b + a * c;
}
pragma(msg, fmadd(6.7, 8.9, 1.3)/* == 17.61f*/);
pragma(msg, fmadd(6.7, 8.9, -1.3)/* == 19.00f*/);
Cheer Guys!
64bit floating point is in!
Hit me with brittle numeric code please!
At the same time 64bit integer support was expanded.
Such that we can now return long and ulong values.
Unfortunately this also broke the phobos unitttests since now
more of is attempted to be evaluated.
Re: CTFE Status 2
On Tue, Jun 20, 2017 at 12:07:00PM +, Stefan Koch via Digitalmars-d wrote:
> On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
> > [ ... ]
>
> limited Support for 32bit floating point ops [+, -, *, /, %] has just
> landed in newCTFE.
Awesome!
> float fmadd(float a, float b, float c)
> {
> return b + a * c;
> }
>
> pragma(msg, fmadd(6.7, 8.9, 1.3)/* == 17.61f*/);
> pragma(msg, fmadd(6.7, 8.9, -1.3)/* == 19.00f*/);
Awesome stuff. Can't wait to get my hands on the newCTFE engine for my
math code!
T
--
Written on the window of a clothing store: No shirt, no shoes, no service.
Re: CTFE Status 2
On Tuesday, 20 June 2017 at 12:07:00 UTC, Stefan Koch wrote: limited Support for 32bit floating point ops [+, -, *, /, %] has just landed in newCTFE. Nice!
Re: CTFE Status 2
On Tuesday, 20 June 2017 at 12:07:00 UTC, Stefan Koch wrote:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch
wrote:
[ ... ]
limited Support for 32bit floating point ops [+, -, *, /, %]
has just landed in newCTFE.
float fmadd(float a, float b, float c)
{
return b + a * c;
}
pragma(msg, fmadd(6.7, 8.9, 1.3)/* == 17.61f*/);
pragma(msg, fmadd(6.7, 8.9, -1.3)/* == 19.00f*/);
These will pass with newCTFE:
static assert(fmadd(0x1.acp+2, 0x1.16p+3, 0x1.4cp+0)
== 0x1.168f5cp+4);
static assert(fmadd(0x1.acp+2, 0x1.16p+3, -0x1.4cp+0)
== -0x1.47a8p-7);
Proving that newCTFE's floating-point-math works the same as
runtime-floating-point math.
Which is not surprising since the interpreter uses same code as
the runtime version.
At-least if you bootstrap dmd with itself.
Re: CTFE Status 2
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
[ ... ]
limited Support for 32bit floating point ops [+, -, *, /, %] has
just landed in newCTFE.
float fmadd(float a, float b, float c)
{
return b + a * c;
}
pragma(msg, fmadd(6.7, 8.9, 1.3)/* == 17.61f*/);
pragma(msg, fmadd(6.7, 8.9, -1.3)/* == 19.00f*/);
Re: CTFE Status 2
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote: [ ... ] newCTFE specific tests are now included in my version of the test-suite. No more silent breakage. (Atleast for anything covered by the test.) For this to work I introduced a __trait which returns true if newCTFE bailed-out on something, and false otherwise. So far it seems to work quite well.
Re: CTFE Status 2
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote: [ ... ] Slice Assignment bugs fixed! With that we are green again. I am going to improve concat a little such that it computes the buffer lengths if it can. And allocs the needed amount upfront. The alternative is lazy concat which is much more work. With alot of potential for bugs. || and && still have me puzzledw :)
Re: CTFE Status 2
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote: [ ... ] Hi there, I just pulled another all nighter. I found a bug in the code that was supposed to adjust the values of || and &&. As will as a mixup in the error messages for overlapping slice-assignment. Both are fixed. I am aware that overlapping slice assignment check is not yet good enough. This is on my short-term todo list. But in the even shorter-term, there are a couple hours sleep waiting to be claimed. Night guys, Stefan
Re: CTFE Status 2
On Tue, Jun 06, 2017 at 03:35:11PM -0400, Steven Schveighoffer via Digitalmars-d wrote: [...] > I can characterize memory corruption bugs as errors that occur > randomly and can manifest in any kind of behavior. Even more nasty is > sometimes they happen in code that you didn't even touch, because it > was *always* happening, but just didn't cause a bug until you changed > memory organization around slightly. [...] Yep, that's exactly the kind of bug I'm talking about. The kind that can appear/disappear depending on the order you link your object files, or declare dummy variables in an unrelated function, or whether you compile with debugging symbols (because that changes the memory layout of your program enough to make the symptoms disappear). The latter is the worst variant of its kind, because it means you're up Bug Creek without any debugging symbol paddles to help you. (And yes, I've actually had to deal with that before. It was not pretty.) T -- "If you're arguing, you're losing." -- Mike Thomas
Re: CTFE Status 2
On 6/6/17 2:51 PM, H. S. Teoh via Digitalmars-d wrote: On Tue, Jun 06, 2017 at 02:23:59PM -0400, Steven Schveighoffer via Digitalmars-d wrote: On 6/6/17 12:39 PM, H. S. Teoh via Digitalmars-d wrote: On Tue, Jun 06, 2017 at 02:03:46AM +, jmh530 via Digitalmars-d wrote: On Tuesday, 6 June 2017 at 00:46:00 UTC, Stefan Koch wrote: Time to find this: roughly 2 weeks. Damn. That's some commitment. 2 weeks is not bad for subtle bugs in complex code like this one. In my day job I've seen bugs that took 2 *months* to figure out. One of them involved a rare race condition that can only be reproduced under very specific circumstances, and it took a long time and a lot of guesswork before a coworker and myself discovered the exact combination that triggered the bug, thereby leading to the subtle problem in a piece of code that looked perfectly innocuous before then. Oh, I've had those before. I had a race condition that reproduced *randomly* and usually took about 2 weeks to happen, and that's by pounding it non-stop. The result was deadlock. Any debugging after the fact resulted in no clues. Only way I solved it was to print out state as it was going, so I could see what happened when the state went bad. I think it took at least 2 cycles to find it. This kind of stuff makes you appreciate how important avoiding race conditions and memory corruption is. [...] Yeah, race conditions and memory corruption / pointer bugs are the worst to track down. Since the codebase I deal with is in C, there are plenty of opportunities for slip-ups that lead to pointer bugs. And the worst of them are dangling pointers... you write to them, and there's no SEGV because they point to valid memory, but that memory has been allocated to something else now. By the time the corruption manifests itself, you're already long, long past the original buggy code, usually in some completely-innocent code that you can stare at for weeks or months and not find a single flaw. Meanwhile the original bug randomly corrupts different things depending on who gets the memory pointed to by the bad pointer, making it almost impossible to reproduce. Even after you reproduce it, you've no idea how to trace it to the original cause, because you're long past where it happened. And it's almost impossible to narrow it down, because reducing the test case may make the bad pointer corrupt something else that you don't see, so you don't know if the bug is still happening or not. Things like this make you *really* appreciate D features like bounds checking and the oft-maligned but life-saving GC. Yep, there were memory errors too. We used a proprietary tool that was like valgrind (this was before valgrind existed) called purify to find those. I think most of them were either double-freeing/deleting something (usually in a destructor that was called more than once -- always set your members that you deleted to null), or freeing new'd memory/deleting malloc'd memory. Thought I had everything, and then the hang. At first we thought it was a memory issue not caught by purify, but then we found it eventually later as I described. I can characterize memory corruption bugs as errors that occur randomly and can manifest in any kind of behavior. Even more nasty is sometimes they happen in code that you didn't even touch, because it was *always* happening, but just didn't cause a bug until you changed memory organization around slightly. Race conditions are also generally random but usually manifest the same way. Both are nasty to find and debug. I don't miss those days :) -Steve
Re: CTFE Status 2
On Tuesday, 6 June 2017 at 18:51:58 UTC, H. S. Teoh wrote: Things like this make you *really* appreciate D features and the oft-maligned but life-saving GC. +1000! GC is an opportunity, not a burden! :-P /Paolo
Re: CTFE Status 2
On Tue, Jun 06, 2017 at 02:23:59PM -0400, Steven Schveighoffer via Digitalmars-d wrote: > On 6/6/17 12:39 PM, H. S. Teoh via Digitalmars-d wrote: > > On Tue, Jun 06, 2017 at 02:03:46AM +, jmh530 via Digitalmars-d wrote: > > > On Tuesday, 6 June 2017 at 00:46:00 UTC, Stefan Koch wrote: > > > > > > > > Time to find this: roughly 2 weeks. > > > > > > > > > > Damn. That's some commitment. > > > > 2 weeks is not bad for subtle bugs in complex code like this one. In > > my day job I've seen bugs that took 2 *months* to figure out. One of > > them involved a rare race condition that can only be reproduced > > under very specific circumstances, and it took a long time and a lot > > of guesswork before a coworker and myself discovered the exact > > combination that triggered the bug, thereby leading to the subtle > > problem in a piece of code that looked perfectly innocuous before > > then. > > Oh, I've had those before. I had a race condition that reproduced > *randomly* and usually took about 2 weeks to happen, and that's by > pounding it non-stop. The result was deadlock. Any debugging after the > fact resulted in no clues. > > Only way I solved it was to print out state as it was going, so I > could see what happened when the state went bad. I think it took at > least 2 cycles to find it. > > This kind of stuff makes you appreciate how important avoiding race > conditions and memory corruption is. [...] Yeah, race conditions and memory corruption / pointer bugs are the worst to track down. Since the codebase I deal with is in C, there are plenty of opportunities for slip-ups that lead to pointer bugs. And the worst of them are dangling pointers... you write to them, and there's no SEGV because they point to valid memory, but that memory has been allocated to something else now. By the time the corruption manifests itself, you're already long, long past the original buggy code, usually in some completely-innocent code that you can stare at for weeks or months and not find a single flaw. Meanwhile the original bug randomly corrupts different things depending on who gets the memory pointed to by the bad pointer, making it almost impossible to reproduce. Even after you reproduce it, you've no idea how to trace it to the original cause, because you're long past where it happened. And it's almost impossible to narrow it down, because reducing the test case may make the bad pointer corrupt something else that you don't see, so you don't know if the bug is still happening or not. Things like this make you *really* appreciate D features like bounds checking and the oft-maligned but life-saving GC. T -- Error: Keyboard not attached. Press F1 to continue. -- Yoon Ha Lee, CONLANG
Re: CTFE Status 2
On 6/6/17 12:39 PM, H. S. Teoh via Digitalmars-d wrote: On Tue, Jun 06, 2017 at 02:03:46AM +, jmh530 via Digitalmars-d wrote: On Tuesday, 6 June 2017 at 00:46:00 UTC, Stefan Koch wrote: Time to find this: roughly 2 weeks. Damn. That's some commitment. 2 weeks is not bad for subtle bugs in complex code like this one. In my day job I've seen bugs that took 2 *months* to figure out. One of them involved a rare race condition that can only be reproduced under very specific circumstances, and it took a long time and a lot of guesswork before a coworker and myself discovered the exact combination that triggered the bug, thereby leading to the subtle problem in a piece of code that looked perfectly innocuous before then. Oh, I've had those before. I had a race condition that reproduced *randomly* and usually took about 2 weeks to happen, and that's by pounding it non-stop. The result was deadlock. Any debugging after the fact resulted in no clues. Only way I solved it was to print out state as it was going, so I could see what happened when the state went bad. I think it took at least 2 cycles to find it. This kind of stuff makes you appreciate how important avoiding race conditions and memory corruption is. -Steve
Re: CTFE Status 2
On Tuesday, 6 June 2017 at 16:39:07 UTC, H. S. Teoh wrote: On Tue, Jun 06, 2017 at 02:03:46AM +, jmh530 via Digitalmars-d wrote: On Tuesday, 6 June 2017 at 00:46:00 UTC, Stefan Koch wrote: > > Time to find this: roughly 2 weeks. > Damn. That's some commitment. 2 weeks is not bad for subtle bugs in complex code like this one. In my day job I've seen bugs that took 2 *months* to figure out. One of them involved a rare race condition that can only be reproduced under very specific circumstances, and it took a long time and a lot of guesswork before a coworker and myself discovered the exact combination that triggered the bug, thereby leading to the subtle problem in a piece of code that looked perfectly innocuous before then. T Wow, 2 Months. And I always feel slow, when a bug takes more then a week. Luckily my architecture is designed to be completely deterministic and reproducibly. So things like race conditions cannot hit me. ... Thank god for that.
Re: CTFE Status 2
On Tue, Jun 06, 2017 at 02:03:46AM +, jmh530 via Digitalmars-d wrote: > On Tuesday, 6 June 2017 at 00:46:00 UTC, Stefan Koch wrote: > > > > Time to find this: roughly 2 weeks. > > > > Damn. That's some commitment. 2 weeks is not bad for subtle bugs in complex code like this one. In my day job I've seen bugs that took 2 *months* to figure out. One of them involved a rare race condition that can only be reproduced under very specific circumstances, and it took a long time and a lot of guesswork before a coworker and myself discovered the exact combination that triggered the bug, thereby leading to the subtle problem in a piece of code that looked perfectly innocuous before then. T -- A mathematician is a device for turning coffee into theorems. -- P. Erdos
Re: CTFE Status 2
On Tuesday, 6 June 2017 at 04:11:33 UTC, Stefan Koch wrote: On Tuesday, 6 June 2017 at 02:03:46 UTC, jmh530 wrote: On Tuesday, 6 June 2017 at 00:46:00 UTC, Stefan Koch wrote: Time to find this: roughly 2 weeks. Damn. That's some commitment. There is no other way, really. These things need to be fixed. Great work. Keep up.
Re: CTFE Status 2
On Tuesday, 6 June 2017 at 02:03:46 UTC, jmh530 wrote: On Tuesday, 6 June 2017 at 00:46:00 UTC, Stefan Koch wrote: Time to find this: roughly 2 weeks. Damn. That's some commitment. There is no other way, really. These things need to be fixed.
Re: CTFE Status 2
On Tuesday, 6 June 2017 at 00:46:00 UTC, Stefan Koch wrote: Time to find this: roughly 2 weeks. Damn. That's some commitment.
Re: CTFE Status 2
On Monday, 5 June 2017 at 20:51:03 UTC, Stefan Koch wrote: On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote: [ .. ] Hi Guys, I am still working on the buggy Slice-Assignment. Just now I fixed a bug where an [Bytecode-Assert] would flip it's the condition ... and trigger whenever it was true instead of false. Assertions are one of things which cannot be tested at compiletime since they need to interact with dmd bummer. Originally I intended for floating-point support to be implemented in this and the following month. Looks like I am not gonna make it. My main system is waiting for replacement parts. And as you know my laptop cannot link dmd when a gui is active... so that's not great either. Cheers, Stefan I solved the slice assignment case. Now here is a subtle bug for you. As you may know, newCTFE has it's own type-system in order to help make the IR simpler. Later on typed version of the ir gets lowered though until only int32 expressions remain. When processing Slice expression I lowered to early though, and returned an i32 to be interpreted as pointer to a slice descriptor. However in the slice-assignment code I wanted to the the elmementType form the SliceExp, which came back as BCType(invalid). Then I took it the size of it, which came back as 0. (an invalid size for an invalid type). And multiplied the index and length, with it to move to the correct point in the slice-buffer. That comes back as target[0 .. 0] = source[0 .. 0]. So no modification occurs. Time to find this: roughly 2 weeks. Why did it take so long ? Because I thought the ABI was at fault; And spent a lot of time staring on memory dumps :(
Re: CTFE Status 2
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote: [ .. ] Hi Guys, I am still working on the buggy Slice-Assignment. Just now I fixed a bug where an [Bytecode-Assert] would flip it's the condition ... and trigger whenever it was true instead of false. Assertions are one of things which cannot be tested at compiletime since they need to interact with dmd bummer. Originally I intended for floating-point support to be implemented in this and the following month. Looks like I am not gonna make it. My main system is waiting for replacement parts. And as you know my laptop cannot link dmd when a gui is active... so that's not great either. Cheers, Stefan
Re: CTFE Status 2
On Sunday, 28 May 2017 at 22:26:18 UTC, Stefan Koch wrote: On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote: [ ... ] Hi Guys, I just fixed the sliceAssignment! now overlapping assignments are correctly detected. I also re-enabled a bailout on struct-member operation which are not simple integers. Which means I chose to ignore some ABI issues for now With that the dmd unittests are green again. Phobos still fails an I am investigating. The failing phobos test works now. However due to the corrected expression handling, I now encounter cases, which I overlooked before. Thus we now fail the dmd testsuite.
Re: CTFE Status 2
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote: [ ... ] Hi Guys, I just fixed the sliceAssignment! now overlapping assignments are correctly detected. I also re-enabled a bailout on struct-member operation which are not simple integers. Which means I chose to ignore some ABI issues for now With that the dmd unittests are green again. Phobos still fails an I am investigating.
Re: CTFE Status 2
On Tuesday, 23 May 2017 at 17:46:23 UTC, Petar Kirov [ZombineDev] wrote: Seems like you need to make -bc-ctfe the default in your dmd branch :P That is what I usually do, however I re-enabled the switch in order to test which error messages the current Interpreter throws out.
Re: CTFE Status 2
On Tuesday, 23 May 2017 at 17:26:01 UTC, Stefan Koch wrote: On Sunday, 21 May 2017 at 14:49:32 UTC, Stefan Koch wrote: Huh apparently I fixed this issue. But I cannot rememberer when or how I did it. This test will now magically work. This concerns me. I finally figured out what was going on. I forgot to activate the -bc-ctfe switch A few days of work into the wrong direction *sigh* Seems like you need to make -bc-ctfe the default in your dmd branch :P
Re: CTFE Status 2
On Sunday, 21 May 2017 at 14:49:32 UTC, Stefan Koch wrote: Huh apparently I fixed this issue. But I cannot rememberer when or how I did it. This test will now magically work. This concerns me. I finally figured out what was going on. I forgot to activate the -bc-ctfe switch A few days of work into the wrong direction *sigh*
Re: CTFE Status 2
On Tuesday, 16 May 2017 at 13:44:27 UTC, Stefan Koch wrote:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch
wrote:
[ ... ]
So ...
I just encountered more ABI issues; related to slices which are
part of structures.
struct R
{
uint[] s1;
uint[] s2;
}
like this.
R returnSlices(int[] s1, int[] s2)
{
return R(s1[], s2[]);
}
static assert(returnSlices([1,2,3], [4,5,6,7]) ==
R([1,2,3][4.5.6.7])); // works
R returnSlicedSlices(int[] s1, int[] s2)
{
return R(s1[], s2[1 .. $-1]);
}
static assert(returnSlicedSlices([1,2,3], [4,5,6,7]) ==
R([1,2,3],[5,6])); // fails
// returns R([1,2,3],null); at the moment
The reason ABI issues.
Where exactly ? No Idea.
Huh apparently I fixed this issue.
But I cannot rememberer when or how I did it.
This test will now magically work.
This concerns me.
Re: CTFE Status 2
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
...
|| works also fine now ... on it's own.
As soon as || and && are mixed wired things happen.
I am happy that the combinations I tried do at least expose the
issue.
This is one of the things that you can totally overlook if the
test-case does not trigger it.
int[2] aaa2(bool b1, bool b2, bool b3, bool b4)
{
int x = 0;
if (b1 && ++x && b2 && x++ && b3 && (b4 || x++))
{
return [x, 1];
}
else
{
return [x, 0];
}
}
static assert(aaa2(1, 0, 1, 0) == [1, 0]);
//static assert(aaa2(1, 1, 1, 0) == [3, 1]); // argh
static assert(aaa2(1, 1, 1, 1) == [2, 1]);
static assert(aaa2(0, 0, 1, 0) == [0, 0]);
int[2] ooo2(bool b1, bool b2, bool b3, bool b4)
{
int x = 0;
if (b1 || x++ || b2 || !x++ || b3 || (b4 && x++))
{
return [x, 1];
}
else
{
return [x, 0];
}
}
static assert(ooo2(1, 0, 1, 0) == [0, 1]);
static assert(ooo2(0, 1, 1, 0) == [1, 1]);
static assert(ooo2(0, 0, 1, 0) == [2, 1]);
static assert(ooo2(0, 0, 0, 0) == [2, 0]);
//static assert(ooo2(0, 0, 0, 1) == [3, 1]); // oh god ...
Re: CTFE Status 2
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
[ ... ]
So I just verified that the following code runs fine with newCTFE.
int[2] aaa2(bool b1, bool b2, bool b3)
{
int x = 0;
if (b1 && ++x && b2 && x++ && b3 && x++)
{
return [x, 1];
}
else
{
return [x, 0];
}
}
static assert(aaa2(1, 0, 1) == [1, 0]);
static assert(aaa2(1, 1, 1) == [3, 1]);
static assert(aaa2(0, 0, 1) == [0, 0]);
Re: CTFE Status 2
On Tuesday, 16 May 2017 at 13:44:27 UTC, Stefan Koch wrote: [ ... ] The reason ABI issues. Where exactly ? No Idea. Not just abi issues ... There are more fundamental problems where we sometimes forget to allocate space, for locals (of composite types).
Re: CTFE Status 2
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
[ ... ]
So ...
I just encountered more ABI issues; related to slices which are
part of structures.
struct R
{
uint[] s1;
uint[] s2;
}
like this.
R returnSlices(int[] s1, int[] s2)
{
return R(s1[], s2[]);
}
static assert(returnSlices([1,2,3], [4,5,6,7]) ==
R([1,2,3][4.5.6.7])); // works
R returnSlicedSlices(int[] s1, int[] s2)
{
return R(s1[], s2[1 .. $-1]);
}
static assert(returnSlicedSlices([1,2,3], [4,5,6,7]) ==
R([1,2,3],[5,6])); // fails
// returns R([1,2,3],null); at the moment
The reason ABI issues.
Where exactly ? No Idea.
Re: CTFE Status 2
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote: [ ... ] So I have fixed a few cases of outer function evaluation. Unfortunately this exposed some hard to track down bugs in how expressions are handled. The JIT and Debugger features are on ice, until those bugs are eliminated. Sorry about the delay.
Re: CTFE Status 2
On Friday, 12 May 2017 at 11:21:56 UTC, Stefan Koch wrote: ... anyway. I am happy this is fixed now. Now I am less happy. The fallout of this fix causes code in std.ascii to miscompile. Apperantly we don't make sure our function list is cleared before finalization.
Re: CTFE Status 2
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
[ ... ]
Hi Guys,
Outer function arguments are now supperted.
meaning this code will now work:
int[] filterBy(int[] arr , bool function(uint) fn)
{
int[] result = [];
uint resultLength;
result.length = arr.length;
foreach(i;0 .. arr.length)
{
auto e = arr[i];
bool r = true;
r = fn(e);
if(r)
{
result[resultLength++] = e;
}
}
int[] filterResult;
filterResult.length = resultLength;
foreach(i; 0 .. resultLength)
{
filterResult[i] = result[i];
}
return filterResult;
}
bool isDiv2(uint e)
{
bool result;
result = (e % 2 == 0);
return result;
}
static assert(filterBy([3,4,5], ) == [4]);
before this would behaved very strangely ;)
because isDiv would have been executed instead filterBy.
And since after bytecode compilation there is no type checking
anymore the arrayPtr would have been interpreter as an integer.
(which is always 4byte aligend)
that would have caused the isDiv to return 1;
which would have been interpreted as address.
and whatever was there would have been treated as array
descriptor.
resulting in mostly the [] return value.
...
anyway.
I am happy this is fixed now.
Re: CTFE Status 2
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote: [ ... ] Thanks to Daniel Murphy's input; '&&' works now in my experimental version. I hope to get it to pass the auto-tester soon! This is a big step :) Thanks Daniel.
Re: CTFE Status 2
On Sunday, 30 April 2017 at 19:52:27 UTC, H. S. Teoh wrote: On Sun, Apr 30, 2017 at 01:26:09PM +, Stefan Koch via Digitalmars-d wrote: On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote: > [ ... ] Big news! The first step to include debug info has been done. Yes this means you will be able to step through ctfe code while the compiler executes it. Wow! Will that be accessible to users in the end? That could be a totally awesome way of debugging CTFE code! I used to think the same, but with each new line of code I write that is to be executed in CT, I become more convinced that there's no need to expose such debugging interface to the end user. Why? Because the end user expects that CTFE either gives exactly the same results as run-time execution or that it stops with an explicit error message on something that is not designed to be executed in CT. Any other problem found during CTFE execution must be 100% reproducible in run time or it's an ICE. Any CTFE debugging should be only for compiler maintainers, and the user shouldn't worry that CTFE could mess up something.
Re: CTFE Status 2
On Wednesday, 3 May 2017 at 07:35:56 UTC, Stefan Koch wrote: On Wednesday, 3 May 2017 at 06:10:22 UTC, Adrian Matoga wrote: So you're going to reinvent TCP in your debugging protocol? No. there is no need for a full blown recovery mechanism. For the typical usecase a lossless orderd connection can be assumed. And most things are not order dependent The debugger isn't a massive, real-time system that needs to service thousands of clients and squeeze as much performance out of the network as possible. The overhead incurred by TCP is essentially not worth considering for something that's going to be run over localhost 90%, and service just the one client. Reinventing the wheel adds a far bigger overhead: Maintaining your new protocol. TCP implementations are readily available. They're well maintained, well documented, and are essentially guaranteed to work across platforms. Implementing a new protocol just adds an extra point of breakage for little to no gain. It also incurs the cost of the associated development time, and - down the line - any time spent fixing or iterating. Not to mention that tests need to be written, documentation needs to be put in place. A classic case of premature optimization.
Re: CTFE Status 2
On Wednesday, 3 May 2017 at 07:35:56 UTC, Stefan Koch wrote: For the typical usecase a lossless orderd connection can be assumed. - udp is not connection oriented, i.e. there is no connection - udp is not lossless and pretending it is means setting yourself up for a headache down the road - udp datagrams are not guaranteed to arrive in the order they were sent and pretending they do is also setting yourself up for a headache down the road What you've described so far is a classic, textbook use case of tcp.
Re: CTFE Status 2
On Wednesday, 3 May 2017 at 04:22:00 UTC, Stefan Koch wrote: [...] I think any ordering should be done explicitly at the debugging protocol level. for example when sending sourceline messages the order is given by the line number and ordering can be done by the application. It's the same for breakpoint setting or for breakpoint trigger notification. Why? If I were to write a client for the debugging protocol, I wouldn't want to write protocol ordering logic (and essentially reimplement part of tcp). I would want to react to protocol messages as they arrive. As for lost packages, we want to respond intelligently as well. The only way I know of to respond intelligently to lost packages using udp - if you care about the information in them (which we do in this use case) - is to implement a retransmit mechanism; i.e. you would be reimplementing another part of tcp. And maybe reduce the amount of data in the package, to make arrival of future packages more likely. You assume a causation between udp datagram size and delivery probability, which - however likely - is an implementation detail.
Re: CTFE Status 2
On Wednesday, 3 May 2017 at 08:23:54 UTC, Nordlöw wrote: On Wednesday, 3 May 2017 at 07:35:56 UTC, Stefan Koch wrote: On Wednesday, 3 May 2017 at 06:10:22 UTC, Adrian Matoga wrote: So you're going to reinvent TCP in your debugging protocol? No. there is no need for a full blown recovery mechanism. For the typical usecase a lossless orderd connection can be assumed. And most things are not order dependent What about packet boundaries? We send source line by line. Packets should be under 1k in most cases.
Re: CTFE Status 2
An article about LLVM jit: https://webkit.org/blog/5852/introducing-the-b3-jit-compiler/
Re: CTFE Status 2
On Sunday, 30 April 2017 at 13:26:09 UTC, Stefan Koch wrote: On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote: [ ... ] Big news! The first step to include debug info has been done. Yes this means you will be able to step through ctfe code while the compiler executes it. This should have been kept secret before C++ steals it and does not give credit to D. :D
Re: CTFE Status 2
On Wednesday, 3 May 2017 at 07:35:56 UTC, Stefan Koch wrote: On Wednesday, 3 May 2017 at 06:10:22 UTC, Adrian Matoga wrote: So you're going to reinvent TCP in your debugging protocol? No. there is no need for a full blown recovery mechanism. For the typical usecase a lossless orderd connection can be assumed. And most things are not order dependent What about packet boundaries?
Re: CTFE Status 2
On Wednesday, 3 May 2017 at 06:10:22 UTC, Adrian Matoga wrote: So you're going to reinvent TCP in your debugging protocol? No. there is no need for a full blown recovery mechanism. For the typical usecase a lossless orderd connection can be assumed. And most things are not order dependent
Re: CTFE Status 2
On Wednesday, 3 May 2017 at 04:22:00 UTC, Stefan Koch wrote: On Tuesday, 2 May 2017 at 22:08:31 UTC, Moritz Maxeiner wrote: [...] Using TCP would just remove any potential future headache from the equation. I think any ordering should be done explicitly at the debugging protocol level. for example when sending sourceline messages the order is given by the line number and ordering can be done by the application. It's the same for breakpoint setting or for breakpoint trigger notification. As for lost packages, we want to respond intelligently as well. And maybe reduce the amount of data in the package, to make arrival of future packages more likely. So you're going to reinvent TCP in your debugging protocol?
Re: CTFE Status 2
On Tuesday, 2 May 2017 at 22:08:31 UTC, Moritz Maxeiner wrote: On Tuesday, 2 May 2017 at 09:55:56 UTC, Stefan Koch wrote: [...] I intended for the debugging functionality to be exposed via a udp socket listening on localhost. Such that a debug-ui does not have to deal with ipc difficulties. Hm, rationale for UDP over TCP here? I would assume one wouldn't want debugging info to be delivered out of order (or not at all); I mean, I guess it would be ok for localhost only (though one is then depending on implementation specifics vs. protocol semantics), but *if* you use sockets, you will eventually get people who use that over the network (and then the fun begins). Using TCP would just remove any potential future headache from the equation. I think any ordering should be done explicitly at the debugging protocol level. for example when sending sourceline messages the order is given by the line number and ordering can be done by the application. It's the same for breakpoint setting or for breakpoint trigger notification. As for lost packages, we want to respond intelligently as well. And maybe reduce the amount of data in the package, to make arrival of future packages more likely.
Re: CTFE Status 2
On Tuesday, 2 May 2017 at 09:55:56 UTC, Stefan Koch wrote: [...] I intended for the debugging functionality to be exposed via a udp socket listening on localhost. Such that a debug-ui does not have to deal with ipc difficulties. Hm, rationale for UDP over TCP here? I would assume one wouldn't want debugging info to be delivered out of order (or not at all); I mean, I guess it would be ok for localhost only (though one is then depending on implementation specifics vs. protocol semantics), but *if* you use sockets, you will eventually get people who use that over the network (and then the fun begins). Using TCP would just remove any potential future headache from the equation.
Re: CTFE Status 2
On Tue, May 02, 2017 at 09:55:56AM +, Stefan Koch via Digitalmars-d wrote: [...] > I intended for the debugging functionality to be exposed via a udp > socket listening on localhost. > Such that a debug-ui does not have to deal with ipc difficulties. > I am strictly against building a debugger into dmd. Nice, that's a good approach. T -- Bomb technician: If I'm running, try to keep up.
Re: CTFE Status 2
On Monday, 1 May 2017 at 19:06:24 UTC, H. S. Teoh wrote: On Mon, May 01, 2017 at 06:23:08PM +, Stefan Koch via Digitalmars-d wrote: [...] I'm not sure about providing a debugger UI inside the compiler itself... it's certainly possible, and could lead to interesting new ways of using a compiler, but I was thinking more along the lines of providing the necessary hooks so that you could attach an external debugger to the CTFE engine. But if the debugger UI is simple enough, perhaps having it built into the compiler may not be a bad thing. It would also avoid potential trouble caused by some platforms not having debuggers capable of plugging into the compiler in that way. But still, I can see people demanding IDE integration for this eventually... :-O T I intended for the debugging functionality to be exposed via a udp socket listening on localhost. Such that a debug-ui does not have to deal with ipc difficulties. I am strictly against building a debugger into dmd.
Re: CTFE Status 2
On Mon, May 01, 2017 at 06:23:08PM +, Stefan Koch via Digitalmars-d wrote: > On Sunday, 30 April 2017 at 19:52:27 UTC, H. S. Teoh wrote: > > On Sun, Apr 30, 2017 at 01:26:09PM +, Stefan Koch via Digitalmars-d > > wrote: > > > On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote: > > > > [ ... ] > > > > > > Big news! The first step to include debug info has been done. > > > > > > Yes this means you will be able to step through ctfe code while > > > the compiler executes it. > > > > Wow! Will that be accessible to users in the end? That could be a > > totally awesome way of debugging CTFE code! > > > > > > T > > Yes the plan is to make it accessible for the advanced user. probably > with a really bad ui, though (since I am awful at UI code). I'm not sure about providing a debugger UI inside the compiler itself... it's certainly possible, and could lead to interesting new ways of using a compiler, but I was thinking more along the lines of providing the necessary hooks so that you could attach an external debugger to the CTFE engine. But if the debugger UI is simple enough, perhaps having it built into the compiler may not be a bad thing. It would also avoid potential trouble caused by some platforms not having debuggers capable of plugging into the compiler in that way. But still, I can see people demanding IDE integration for this eventually... :-O T -- Obviously, some things aren't very obvious.
Re: CTFE Status 2
On Sunday, 30 April 2017 at 19:52:27 UTC, H. S. Teoh wrote: On Sun, Apr 30, 2017 at 01:26:09PM +, Stefan Koch via Digitalmars-d wrote: On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote: > [ ... ] Big news! The first step to include debug info has been done. Yes this means you will be able to step through ctfe code while the compiler executes it. Wow! Will that be accessible to users in the end? That could be a totally awesome way of debugging CTFE code! T Yes the plan is to make it accessible for the advanced user. probably with a really bad ui, though (since I am awful at UI code).
Re: CTFE Status 2
On Sun, Apr 30, 2017 at 01:26:09PM +, Stefan Koch via Digitalmars-d wrote: > On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote: > > [ ... ] > > Big news! > The first step to include debug info has been done. > > Yes this means you will be able to step through ctfe code while the > compiler executes it. Wow! Will that be accessible to users in the end? That could be a totally awesome way of debugging CTFE code! T -- Life begins when you can spend your spare time programming instead of watching television. -- Cal Keegan
Re: CTFE Status 2
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote: [ ... ] Big news! The first step to include debug info has been done. Yes this means you will be able to step through ctfe code while the compiler executes it.
Re: CTFE Status 2
On Friday, 28 April 2017 at 17:53:04 UTC, Stefan Koch wrote:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch
wrote:
[ ... ]
Hi Guys, I just implemented sliceAssigment.
meaning the following code will now compile:
uint[] assignSlice(uint from, uint to, uint[] stuff)
{
uint[] slice;
slice.length = to + 4;
foreach (uint i; 0 .. to + 4)
{
slice[i] = i + 1;
}
slice[from .. to] = stuff;
return slice;
}
static assert(assignSlice(1, 4, [9, 8, 7]) == [1, 9, 8, 7, 5,
6, 7, 8]);
FIXED!
Re: CTFE Status 2
On Friday, 28 April 2017 at 08:47:43 UTC, Stefan Koch wrote:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch
wrote:
[ ... ]
After a little of exploration of the JIT, I have now determined
that a simple risc architecture is still the best.
(codegen for scaled loads is hard :p)
I am now back to fixing non-compiling code,
such as :
struct S
{
uint[] slice;
}
uint fn()
{
S s;
s.slice.length = 12;
return cast(uint)s.slice.length;
}
static assert(fn() == 12);
This simple test does not compile because;
ahm well ...
Somewhere along the road we loose the type of s.slice and we
cannot tell where to get .length from.
I fixed this just now!
ABI's are hard :)
Re: CTFE Status 2
On Friday, 28 April 2017 at 17:53:04 UTC, Stefan Koch wrote:
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch
wrote:
[ ... ]
Hi Guys, I just implemented sliceAssigment.
meaning the following code will now compile:
uint[] assignSlice(uint from, uint to, uint[] stuff)
{
uint[] slice;
slice.length = to + 4;
foreach (uint i; 0 .. to + 4)
{
slice[i] = i + 1;
}
slice[from .. to] = stuff;
return slice;
}
static assert(assignSlice(1, 4, [9, 8, 7]) == [1, 9, 8, 7, 5,
6, 7, 8]);
as always ... I spoke too soon :(
while running test I forgot to specify -bc-ctfe ...
Although I use the same code for slicing ... it seems it
misbehaves in the usecase.
Re: CTFE Status 2
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
[ ... ]
Hi Guys, I just implemented sliceAssigment.
meaning the following code will now compile:
uint[] assignSlice(uint from, uint to, uint[] stuff)
{
uint[] slice;
slice.length = to + 4;
foreach (uint i; 0 .. to + 4)
{
slice[i] = i + 1;
}
slice[from .. to] = stuff;
return slice;
}
static assert(assignSlice(1, 4, [9, 8, 7]) == [1, 9, 8, 7, 5, 6,
7, 8]);
Re: CTFE Status 2
On Friday, 28 April 2017 at 13:13:16 UTC, Stefan Koch wrote: Do you mean no Jit? Of course there will be a JIT. Ah, I misunderstood you formulation. But currently I am fixing busy bugs in the generated IR. So the implementation of jit will have to wait a little. Ok. Thanks.
Re: CTFE Status 2
On Friday, 28 April 2017 at 13:03:42 UTC, Nordlöw wrote: On Friday, 28 April 2017 at 08:47:43 UTC, Stefan Koch wrote: After a little of exploration of the JIT, I have now determined that a simple risc architecture is still the best. (codegen for scaled loads is hard :p) Do you mean no Jit? Of course there will be a JIT. But currently I am fixing busy bugs in the generated IR. So the implementation of jit will have to wait a little.
Re: CTFE Status 2
On Friday, 28 April 2017 at 08:47:43 UTC, Stefan Koch wrote: After a little of exploration of the JIT, I have now determined that a simple risc architecture is still the best. (codegen for scaled loads is hard :p) Do you mean no Jit?
Re: CTFE Status 2
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
[ ... ]
After a little of exploration of the JIT, I have now determined
that a simple risc architecture is still the best.
(codegen for scaled loads is hard :p)
I am now back to fixing non-compiling code,
such as :
struct S
{
uint[] slice;
}
uint fn()
{
S s;
s.slice.length = 12;
return cast(uint)s.slice.length;
}
static assert(fn() == 12);
This simple test does not compile because;
ahm well ...
Somewhere along the road we loose the type of s.slice and we
cannot tell where to get .length from.
Re: CTFE Status 2
On Thursday, 27 April 2017 at 08:51:17 UTC, Dmitry Olshansky wrote: On 4/27/17 4:15 AM, Stefan Koch wrote: On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote: [ ... ] Hi Guys, As you already probably know some work has been done in the past week to get an x86 jit rolling. It is designed to produce very simple code with _any_ optimization at all. Since optimization introduces heavy complexity down the road, even if at first it looks very affordable. My opinion is : "_any_ optimization too much." There is also trade-off of spending too much time doing an optimization. That being said simple peep-hole optimizations may be well worth the effort. This stance should make it possible to get some _really_ shiny performance numbers for dconf. Cheers, Stefan I should probably clarify; I made a typo. I was meaning to write "without _any_ optimization at all." Peep-holing would be worth it for wanting to get the last drop of performance; However in the specific case of newCTFE, the crappiest JIT will already be much faster then an optimized interpreter would be. Small peephole optimization quickly turns into and endless source of bugs.
Re: CTFE Status 2
On 4/27/17 4:15 AM, Stefan Koch wrote: On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote: [ ... ] Hi Guys, As you already probably know some work has been done in the past week to get an x86 jit rolling. It is designed to produce very simple code with _any_ optimization at all. Since optimization introduces heavy complexity down the road, even if at first it looks very affordable. My opinion is : "_any_ optimization too much." There is also trade-off of spending too much time doing an optimization. That being said simple peep-hole optimizations may be well worth the effort. This stance should make it possible to get some _really_ shiny performance numbers for dconf. Cheers, Stefan
Re: CTFE Status 2
On Thursday, 27 April 2017 at 03:33:03 UTC, H. S. Teoh wrote: Is it possible at all to use any of the backend (in particular what parts of the optimizer that are pertinent), or is the API not conducive for this? T It is of course possible to use dmds backend but not very desirable, dmds backend works on an expression-tree, which would be expensive to build from the linear IR newCTFE uses. Dmds backend is also very hard to debug for anyone who is not Walter. CTFE is the common case will be fastest if executed without any optimizer interfering. modern x86 chips done a very fine job indeed executing crappy code fast. Therefore making it possible to get away with very simple and fast codegen. (Where fast means code-generation speed rather then code execution speed).
Re: CTFE Status 2
On Thu, Apr 27, 2017 at 02:15:30AM +, Stefan Koch via Digitalmars-d wrote: > On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote: > > [ ... ] > Hi Guys, > > As you already probably know some work has been done in the past week > to get an x86 jit rolling. > > It is designed to produce very simple code with _any_ optimization at > all. > > Since optimization introduces heavy complexity down the road, even if > at first it looks very affordable. My opinion is : "_any_ optimization > too much." > > This stance should make it possible to get some _really_ shiny > performance numbers for dconf. [...] Is it possible at all to use any of the backend (in particular what parts of the optimizer that are pertinent), or is the API not conducive for this? T -- It always amuses me that Windows has a Safe Mode during bootup. Does that mean that Windows is normally unsafe?
Re: CTFE Status 2
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote: [ ... ] Hi Guys, As you already probably know some work has been done in the past week to get an x86 jit rolling. It is designed to produce very simple code with _any_ optimization at all. Since optimization introduces heavy complexity down the road, even if at first it looks very affordable. My opinion is : "_any_ optimization too much." This stance should make it possible to get some _really_ shiny performance numbers for dconf. Cheers, Stefan
Re: CTFE Status 2
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote: [ ... ] Hi Guys, I just fixed default initialization of structs. So now a larger portion of code will be compiled and executed by newCTFE. my_struct MyStruct; will now work, before it would trigger a bailout. NOTE: this will create bogus results if the struct does contain complex initializers i.e. anything other then integers. Complex type support will come after dconf.
Re: CTFE Status 2
On Saturday, 15 April 2017 at 10:30:57 UTC, Moritz Maxeiner wrote: On Saturday, 15 April 2017 at 10:10:54 UTC, Stefan Koch wrote: On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote: Hi Guys, due to the old CTFE status thread getting to page 30, I am now starting a new one. [...] The llvm backend is back in a fully working state. It's about 2times slower in my then my interpreter ;) Huh. In all cases, or only in trivial ones? Because I would have expected the overhead of jitting to become less relevant the more complex stuff you interpret vs jit. It's an average number. on tests the represent the usual usage of ctfe.
Re: CTFE Status 2
On Saturday, 15 April 2017 at 10:10:54 UTC, Stefan Koch wrote: On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote: Hi Guys, due to the old CTFE status thread getting to page 30, I am now starting a new one. [...] The llvm backend is back in a fully working state. It's about 2times slower in my then my interpreter ;) Huh. In all cases, or only in trivial ones? Because I would have expected the overhead of jitting to become less relevant the more complex stuff you interpret vs jit.
Re: CTFE Status 2
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote: Hi Guys, due to the old CTFE status thread getting to page 30, I am now starting a new one. [...] The llvm backend is back in a fully working state. It's about 2times slower in my then my interpreter ;)
Re: CTFE Status 2
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
{ ... }
Wonderful news!
Most of the Byteocode macros are gone!
meaning less templates and faster bytecode generartion!
Re: CTFE Status 2
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote:
[ ... ]
Hi I want to share another story.
I was pretty happy to have recursive function calls working.
So happy in fact that I overlooked that they were actually
generated twice.
Let me illustrate what happend.
Suppose we have the following module :
void fn(uint rcount)
{
uint ctr = rcount;
while (rcount--)
ctr += fn(rcount);
return ctr;
}
pragma(msg, fn(26));
the compiler hits the pragma(msg) and goes on to evaluate fn(26)
newCTFE receives the functionBody as an ast-node.
it starts processing (function 1)
and hits the fn(rcount)
it checks if it has the code for fn already.
this check returns false since fn has not been completely
generated yet.
when this check returns it will write the FunctionBody in it's
todo list.
it wires up the call to the entry in the todo list. (function 2)
it then hits the end of fn (function 1)and saves it in its
code-cache.
now it processes the TODO list
it finds that it has to process fn.
it starts processing fn (function 2)
it hits the call.
This time it does find an entry in the code cache for fn (funcion
1)
it wires of the call
and returns.
The generate pseudo-code looks like :
... fn_0 (...) {
...
call (fn_1, ...)
...
}
... fn_1 (...) {
...
call (fn_0, ...)
...
}
It was very surprised then I saw this :)
Cheers,
Stefan
Re: CTFE Status 2
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote: [ ... ] Comma expressions should now work.
Re: CTFE Status 2
On Wed, Apr 12, 2017 at 01:18:13PM +, Stefan Koch via Digitalmars-d wrote: > On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote: > > [ ... ] > > I just fixed the static assert((null ~ null) is null); > Hence I can now enable string-concat! [...] Good news! T -- Век живи - век учись. А дураком помрёшь.
Re: CTFE Status 2
On Wednesday, 12 April 2017 at 09:19:39 UTC, Stefan Koch wrote: On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote: [ ... ] I just found more states we get into, that should be impossible to ever get into. I am stumped. Baffled. And seriously befuddled! So .. this is partially because we assume the stack to be zeroed if we have not written to it yet. It is zero-initialized after all, however If we are returning from a function that wrote to the stack and then we are calling another function, that function will see the state the previous function left there... which just means ... we have to zero our temporaries and locals on function entery. implementing this however breaks incremental code-generation. Aw
Re: CTFE Status 2
On Thursday, 16 February 2017 at 21:05:51 UTC, Stefan Koch wrote: [ ... ] I just fixed the static assert((null ~ null) is null); Hence I can now enable string-concat!
