Re: Bye bye, fast compilation times

2018-02-13 Thread Stefan Koch via Digitalmars-d
On Tuesday, 13 February 2018 at 05:47:10 UTC, Dmitry Olshansky 
wrote:


Was once on my together with other OS memory manager functions, 
but postponed the work indefinetly.


https://github.com/dlang/druntime/pull/1549

If someone is willing to revive that I’d gladly assist with 
review.


Lastly on Windows it would need FlushCpuCaches call before 
executing new memory.


And ofc JIT is cool, but it would be more cool to have sane 
interpreter that doesn’t leak sooner. Simply put JIT is x5 work 
due to different architectures and seeing first-hand how it 
goes I’m not sure we want that in our compiler yet.


Since dmd is only targeting x86/x86_64 there is really just one 
arch to support for now.
All the others can fallback to either the interpreter or 
generated c code compiled into a shared lib :)


newCTFE already provides a very low-level IR that should be 
trivially translatable to machine -code. (famous last words :o) )


Re: Bye bye, fast compilation times

2018-02-12 Thread Dmitry Olshansky via Digitalmars-d

On Monday, 12 February 2018 at 00:24:32 UTC, Bastiaan Veelo wrote:
On Wednesday, 7 February 2018 at 09:27:47 UTC, Stefan Koch 
wrote:
One thing that I really need in druntime is a cross-platform 
way to allocate executable memory-pages, this can be done by 
someone else.
Is this on someone's agenda? It probably needs an enhancement 
request at the very least, I don't think it's there yet [1].




Was once on my together with other OS memory manager functions, 
but postponed the work indefinetly.


https://github.com/dlang/druntime/pull/1549

If someone is willing to revive that I’d gladly assist with 
review.


Lastly on Windows it would need FlushCpuCaches call before 
executing new memory.


And ofc JIT is cool, but it would be more cool to have sane 
interpreter that doesn’t leak sooner. Simply put JIT is x5 work 
due to different architectures and seeing first-hand how it goes 
I’m not sure we want that in our compiler yet.






Re: Bye bye, fast compilation times

2018-02-11 Thread Bastiaan Veelo via Digitalmars-d

On Wednesday, 7 February 2018 at 09:27:47 UTC, Stefan Koch wrote:
One thing that I really need in druntime is a cross-platform 
way to allocate executable memory-pages, this can be done by 
someone else.
Is this on someone's agenda? It probably needs an enhancement 
request at the very least, I don't think it's there yet [1].


Another Thing that can be done is reviewing the code and 
alerting me to potential problems. i.e. Missing or 
indecipherable comments as well as spelling mistakes.

I had a go at this [2].

[1] 
https://issues.dlang.org/buglist.cgi?bug_severity=enhancement_status=NEW_status=ASSIGNED_status=REOPENED=druntime_id=219522=D_format=advanced

[2] https://github.com/UplinkCoder/dmd/pull/3


Re: Bye bye, fast compilation times

2018-02-11 Thread aliak via Digitalmars-d

On Sunday, 11 February 2018 at 16:35:35 UTC, Adam D. Ruppe wrote:
The isemail function isn't about validating email addresses. It 
is just about recognizing something that looks like one. just 
like isurl doesn't actually try to fetch the site to see if it 
is broken, it just sees if it looks like one as a first step.


*valid email format... (is better? :) )

When someone says isurl checks if a string is a valid url, I 
don't think the general assumption is that it makes a network 
call to check if it is a resolvable url. (could be mistaken of 
course, but not to me at least). Isurl checks that the format is 
correct. Same for isemail. The isemail API and the docs all use 
the term valid as well.


Plus, to further see how hard it is to validate an email, these 
are apparently all erroneous results (granted wikipedia could be 
wrong as well):


import std.net.isemail, std.stdio;

void main() {
isEmail("john.smith(comment)@example.com").valid.writeln; // 
is valid, prints false
isEmail("user@[2001:DB8::1]").valid.writeln; // is valid, 
prints false
isEmail(`" "@example.org`).valid.writeln; // not valid, 
prints true

isEmail(`"very.unusu...@.unusual.com"@example.com`).valid.writeln; // not valid, prints true

}



Re: Bye bye, fast compilation times

2018-02-11 Thread Adam D. Ruppe via Digitalmars-d

On Sunday, 11 February 2018 at 16:26:19 UTC, aliak wrote:
If anyone wants to do email validation this should be read 
first:


The isemail function isn't about validating email addresses. It 
is just about recognizing something that looks like one. just 
like isurl doesn't actually try to fetch the site to see if it is 
broken, it just sees if it looks like one as a first step.


Re: Bye bye, fast compilation times

2018-02-11 Thread aliak via Digitalmars-d

On Friday, 9 February 2018 at 14:19:56 UTC, bauss wrote:
Generally the best way to validate an email is just to check if 
there is a value before @ and a value after.


The real way to validate an email is to check if the email 
exists on a SMTP server, BUT some SMTP servers will not provide 
such information (Such as gmail I think?) and thus you can't 
really rely on that either.


+1.

If anyone wants to do email validation this should be read first:

https://hackernoon.com/the-100-correct-way-to-validate-email-addresses-7c4818f24643



Re: Bye bye, fast compilation times

2018-02-11 Thread Nathan S. via Digitalmars-d

On Wednesday, 7 February 2018 at 08:21:01 UTC, Seb wrote:

There's `version(StdUnittest)` since a few days


And I've just submitted a pull request that adds 
`version(StdUnittest)` to every unittest in the standard library.


https://github.com/dlang/phobos/pull/6159


Re: Bye bye, fast compilation times

2018-02-09 Thread bauss via Digitalmars-d
On Tuesday, 6 February 2018 at 20:30:42 UTC, Steven Schveighoffer 
wrote:


The regex in question I think is to ensure an email address 
like abc@192.168.0.5 has a valid IP address. The D1 function 
doesn't support that requirement.




-Steve


An invalid IP is not necessarily an invalid email though.

You'd be surprised how much __garbage__ a valid email actually 
can contain.


https://www.w3.org/Protocols/rfc822/

Generally the best way to validate an email is just to check if 
there is a value before @ and a value after.


The real way to validate an email is to check if the email exists 
on a SMTP server, BUT some SMTP servers will not provide such 
information (Such as gmail I think?) and thus you can't really 
rely on that either.


Re: Bye bye, fast compilation times

2018-02-08 Thread Nick Sabalausky (Abscissa) via Digitalmars-d

On 02/08/2018 06:21 AM, Martin Tschierschke wrote:


Beginner question:
How to split my project, to compile the regex part separately as a lib 
and just link them?




Unfortunately that depends completely on what buildsystem you're using. 
But if you're just calling the compiler directly, then it's really easy:


> dmd -lib -of=myLib.a [all other flags your project may need] 
fileYouWantInLib.d anyOtherFileYouAlsoWant.d


> dmd myLib.a [your project's usual flags, and all the rest of your .d 
files]


If on windows, then just replace ".a" with ".lib".


Re: Bye bye, fast compilation times

2018-02-08 Thread H. S. Teoh via Digitalmars-d
On Wed, Feb 07, 2018 at 10:59:51PM -0500, Nick Sabalausky (Abscissa) via 
Digitalmars-d wrote:
[...]
> On 02/06/2018 08:47 PM, jmh530 wrote:
> > 
> > Would it help to take the approach of mir, i.e. putting
> > version(mir_test) before all the unittests?
> 
> That used to be a very common idiom. (And I agree with Jonathan M
> Davis: it's a STRONG sign the current design needs fixed). But newer
> versions of dub, intelligently, will only compile the files in the
> main project with -unittest, which solves the problem...*for dub
> users*. Unfortunately, this means that the idiom above has become less
> common and libraries have become less usable for anyone using a build
> tool *other* than dub. :(

Actually, it does *not* solve the problem for dub users.  Compiling with
-unittest causes unittests for imported modules to be instantiated too.
(Otherwise, Phobos unittests wouldn't show up in a -unittest build since
we never compile Phobos modules directly in a user project!)


T

-- 
ASCII stupid question, getty stupid ANSI.


Re: Bye bye, fast compilation times

2018-02-08 Thread Martin Tschierschke via Digitalmars-d

On Monday, 5 February 2018 at 21:27:57 UTC, H. S. Teoh wrote:
One of my D projects for the past while has been taking 
unusually long times to compile.  This morning, I finally 
decided to sit down and figure out exactly why. What I found 
was rather disturbing:


--
import std.regex;
void main() {
auto re = regex(``);
}
--

Compile command: time dmd -c test.d

Output:
--
real0m3.113s
user0m2.884s
sys 0m0.226s
--

Comment out the call to `regex()`, and I get:

--
real0m0.285s
user0m0.262s
sys 0m0.023s
--

Clearly, something is wrong if the mere act of compiling a 
regex causes a 4-line program to take *3 seconds* to compile, 
where normally dmd takes less than a second.



Thank you for this finding!

I was wondering why my little vibe.d project started to take 
approximately twice the
time to compile, and because of making a mistake in my test 
setup, even my minimal
program still included the file containing the regex. So that 
even reducing the used
code to a minimum the compilation time was ~7 sec compared to 
less than 4 seconds.


Would be cool if we could get fast compilation of regex.

I am coming from using scripting languages (perl and ruby) using 
regex a lot, so that this is really disappointing for me.


Beginner question:
How to split my project, to compile the regex part separately as 
a lib and just link them?




Re: Bye bye, fast compilation times

2018-02-08 Thread Stefan Koch via Digitalmars-d
On Wednesday, 7 February 2018 at 22:00:48 UTC, Bastiaan Veelo 
wrote:
On Wednesday, 7 February 2018 at 09:27:47 UTC, Stefan Koch 
wrote:
Another Thing that can be done is reviewing the code and 
alerting me to potential problems. i.e. Missing or 
indecipherable comments as well as spelling mistakes.
(with the correction please (just telling me something is 
wrong, will not help since I obliviously don't know how to 
spell it))


What is the preferred place for this? 
https://github.com/dlang/dmd/pull/7073 or do you want PRs 
against a fork of yours?

Corrected link:
https://github.com/UplinkCoder/dmd/tree/newCTFE_reboot


Re: Bye bye, fast compilation times

2018-02-08 Thread Stefan Koch via Digitalmars-d
On Wednesday, 7 February 2018 at 22:00:48 UTC, Bastiaan Veelo 
wrote:
On Wednesday, 7 February 2018 at 09:27:47 UTC, Stefan Koch 
wrote:
Another Thing that can be done is reviewing the code and 
alerting me to potential problems. i.e. Missing or 
indecipherable comments as well as spelling mistakes.
(with the correction please (just telling me something is 
wrong, will not help since I obliviously don't know how to 
spell it))


What is the preferred place for this? 
https://github.com/dlang/dmd/pull/7073 or do you want PRs 
against a fork of yours?


I'd prefer a pr against 
https://github.com/UplinkCoder/dmd/newCTFE_reboot


Re: Bye bye, fast compilation times

2018-02-07 Thread Nick Sabalausky (Abscissa) via Digitalmars-d
Yes. This has been a personal annoyance for many years. Even tried 
arguing some time back to get it fixed to no avail. Really hoping for 
better success this time.


On 02/06/2018 08:47 PM, jmh530 wrote:


Would it help to take the approach of mir, i.e. putting 
version(mir_test) before all the unittests?


That used to be a very common idiom. (And I agree with Jonathan M Davis: 
it's a STRONG sign the current design needs fixed). But newer versions 
of dub, intelligently, will only compile the files in the main project 
with -unittest, which solves the problem...*for dub users*. 
Unfortunately, this means that the idiom above has become less common 
and libraries have become less usable for anyone using a build tool 
*other* than dub. :(


Re: Bye bye, fast compilation times

2018-02-07 Thread Walter Bright via Digitalmars-d

On 2/7/2018 1:07 PM, Nathan S. wrote:

On Tuesday, 6 February 2018 at 22:29:07 UTC, Walter Bright wrote:

nobody uses regex for lexer in a compiler.


Some years ago I was surprised when I saw this in Clojure's source code. It 
appears to still be there today:


https://github.com/clojure/clojure/blob/1215ba346ffea3fe48def6ec70542e3300b6f9ed/src/jvm/clojure/lang/LispReader.java#L66-L73 



---
static Pattern symbolPat = 
Pattern.compile("[:]?([\\D&&[^/]].*/)?(/|[\\D&&[^/]][^/]*)");
//static Pattern varPat = 
Pattern.compile("([\\D&&[^:\\.]][^:\\.]*):([\\D&&[^:\\.]][^:\\.]*)");

//static Pattern intPat = Pattern.compile("[-+]?[0-9]+\\.?");
static Pattern intPat =
     Pattern.compile(
 
"([-+]?)(?:(0)|([1-9][0-9]*)|0[xX]([0-9A-Fa-f]+)|0([0-7]+)|([1-9][0-9]?)[rR]([0-9A-Za-z]+)|0[0-9]+)(N)?"); 


static Pattern ratioPat = Pattern.compile("([-+]?[0-9]+)/([0-9]+)");
static Pattern floatPat = 
Pattern.compile("([-+]?[0-9]+(\\.[0-9]*)?([eE][-+]?[0-9]+)?)(M)?");

---



Yes, I'm sure somebody does it. And now that regex has produced a match, you 
have to scan it again to turn it into a number, making for slow lexing. And if 
regex doesn't produce a match, you get a generic error message rather than 
something specific like "character 'A' is not allowed in a numeric literal".


(Generic error messages are one of the downsides of using tools like lex and 
yacc.)


Re: Bye bye, fast compilation times

2018-02-07 Thread Bastiaan Veelo via Digitalmars-d

On Wednesday, 7 February 2018 at 09:27:47 UTC, Stefan Koch wrote:
Another Thing that can be done is reviewing the code and 
alerting me to potential problems. i.e. Missing or 
indecipherable comments as well as spelling mistakes.
(with the correction please (just telling me something is 
wrong, will not help since I obliviously don't know how to 
spell it))


What is the preferred place for this? 
https://github.com/dlang/dmd/pull/7073 or do you want PRs against 
a fork of yours?


Re: Bye bye, fast compilation times

2018-02-07 Thread Nathan S. via Digitalmars-d

On Tuesday, 6 February 2018 at 22:29:07 UTC, Walter Bright wrote:

nobody uses regex for lexer in a compiler.


Some years ago I was surprised when I saw this in Clojure's 
source code. It appears to still be there today:


https://github.com/clojure/clojure/blob/1215ba346ffea3fe48def6ec70542e3300b6f9ed/src/jvm/clojure/lang/LispReader.java#L66-L73

---
static Pattern symbolPat = 
Pattern.compile("[:]?([\\D&&[^/]].*/)?(/|[\\D&&[^/]][^/]*)");
//static Pattern varPat = 
Pattern.compile("([\\D&&[^:\\.]][^:\\.]*):([\\D&&[^:\\.]][^:\\.]*)");

//static Pattern intPat = Pattern.compile("[-+]?[0-9]+\\.?");
static Pattern intPat =
Pattern.compile(

"([-+]?)(?:(0)|([1-9][0-9]*)|0[xX]([0-9A-Fa-f]+)|0([0-7]+)|([1-9][0-9]?)[rR]([0-9A-Za-z]+)|0[0-9]+)(N)?");
static Pattern ratioPat = 
Pattern.compile("([-+]?[0-9]+)/([0-9]+)");
static Pattern floatPat = 
Pattern.compile("([-+]?[0-9]+(\\.[0-9]*)?([eE][-+]?[0-9]+)?)(M)?");

---



Re: Bye bye, fast compilation times

2018-02-07 Thread Nicholas Wilson via Digitalmars-d
On Wednesday, 7 February 2018 at 02:05:46 UTC, Timothee Cour 
wrote:
how about using same syntax (and reusing logic) as newly 
introduced:

` -i=+foo.bar,+baz-baz.bad`

`dmd -unittest=+foo.bar,+baz,-baz.bad rest_of_arguments`

which would only enable unittests as specified? It's flexible 
and intuitive


I like it! That only leaves a question of defaults.


Re: Bye bye, fast compilation times

2018-02-07 Thread Stefan Koch via Digitalmars-d

On Tuesday, 6 February 2018 at 18:56:44 UTC, H. S. Teoh wrote:


We seriously need to get newCTFE finished and merged.  Stefan 
is very busy with other stuff ATM; I wonder if a few of us can 
continue his work and get newCTFE into a mergeable state.  
Given how much D's "compile-time" features are advertised, and 
D's new (ick) slogan of being fast or whatever, it's high time 
we actually delivered on our promises by actually making CTFE 
more usable.



There are some good news for you.
I've recently allocated a few more resources to newCTFE again.

I have to stress that it is not enough to get newCTFE feature 
complete.

It is also vital make performance-related pass through the code.
newCTFE currently still at a Proof-Of-Concept quality level.

That said, newCTFE is designed with performance and JIT in mind.
It can achieve a 10-30x speed-up when implemented properly.

One thing that I really need in druntime is a cross-platform way 
to allocate executable memory-pages, this can be done by someone 
else.


Another Thing that can be done is reviewing the code and alerting 
me to potential problems. i.e. Missing or indecipherable comments 
as well as spelling mistakes.
(with the correction please (just telling me something is wrong, 
will not help since I obliviously don't know how to spell it))





Re: Bye bye, fast compilation times

2018-02-07 Thread Stefan Koch via Digitalmars-d

On Tuesday, 6 February 2018 at 18:56:44 UTC, H. S. Teoh wrote:


We seriously need to get newCTFE finished and merged.  Stefan 
is very busy with other stuff ATM; I wonder if a few of us can 
continue his work and get newCTFE into a mergeable state.  
Given how much D's "compile-time" features are advertised, and 
D's new (ick) slogan of being fast or whatever, it's high time 
we actually delivered on our promises by actually making CTFE 
more usable.



There are some good news for you.
I've recently allocated a few more resources to newCTFE again.

I have to stress that it is not enough to get newCTFE feature 
complete.

It is also vital make performance-related pass through the code.
newCTFE currently still at a Proof-Of-Concept quality level.

That said, newCTFE is designed with performance and JIT in mind.
It can achieve a 10-30x speed-up when implemented properly.

One thing that I really need in druntime is a cross-platform way 
to allocate executable memory-pages, this can be done by someone 
else.


Another Thing that can be done is reviewing the code and alerting 
me to potential problems. i.e. Missing or indecipherable comments 
as well as spelling mistakes.
(with the correction please (just telling me something is wrong, 
will not help since I obliviously don't know how to spell it))





Re: Bye bye, fast compilation times

2018-02-07 Thread Seb via Digitalmars-d

On Wednesday, 7 February 2018 at 01:20:04 UTC, H. S. Teoh wrote:
On Mon, Feb 05, 2018 at 01:27:57PM -0800, H. S. Teoh via 
Digitalmars-d wrote:

[...]

[...]

I don't want this thread to turn into ragging on std.regex, so 
here's the next instalment of this saga.


[...]


Already done. There's `version(StdUnittest)` since a few days:

https://github.com/dlang/phobos/pull/5927
https://github.com/dlang/phobos/pull/6107


Re: Bye bye, fast compilation times

2018-02-06 Thread Walter Bright via Digitalmars-d

On 2/6/2018 2:51 PM, Steven Schveighoffer wrote:

The regex problem is being solved:

https://github.com/dlang/phobos/pull/6129


Great!



Re: Bye bye, fast compilation times

2018-02-06 Thread Timothee Cour via Digitalmars-d
how about using same syntax (and reusing logic) as newly introduced:
` -i=+foo.bar,+baz-baz.bad`

`dmd -unittest=+foo.bar,+baz,-baz.bad rest_of_arguments`

which would only enable unittests as specified? It's flexible and intuitive

On Tue, Feb 6, 2018 at 5:56 PM, Jonathan M Davis via Digitalmars-d
 wrote:
> On Wednesday, February 07, 2018 01:47:19 jmh530 via Digitalmars-d wrote:
>> On Wednesday, 7 February 2018 at 01:20:04 UTC, H. S. Teoh wrote:
>> > So I'd like to propose that we do something similar to what we
>> > did with template instantiations a couple of years ago: make it
>> > so that unittests are only instantiated if the module they
>> > occur in is being compiled, otherwise ignore them (even in the
>> > face of -unittest).  This way, adding unittests to Phobos won't
>> > cause unintentional slowdowns / unittest bloat across *all* D
>> > projects that import the affected Phobos modules. (Seen from
>> > this angle, it's a pretty hefty cost.)
>>
>> Would it help to take the approach of mir, i.e. putting
>> version(mir_test) before all the unittests?
>
> It would, but if we have to do that in front of all unittest blocks in a
> library, I think that that's a strong sign that the current design has a
> serious problem that should be fixed if possible.
>
> - Jonathan M Davis
>


Re: Bye bye, fast compilation times

2018-02-06 Thread psychoticRabbit via Digitalmars-d

On Tuesday, 6 February 2018 at 20:11:56 UTC, Walter Bright wrote:


std.string.isEmail() in D1 was a simple function. Maybe regex 
is just the wrong solution for this problem.


[...]



C .. D style. I love it! (bugs and all).


Re: Bye bye, fast compilation times

2018-02-06 Thread Jonathan M Davis via Digitalmars-d
On Wednesday, February 07, 2018 01:47:19 jmh530 via Digitalmars-d wrote:
> On Wednesday, 7 February 2018 at 01:20:04 UTC, H. S. Teoh wrote:
> > So I'd like to propose that we do something similar to what we
> > did with template instantiations a couple of years ago: make it
> > so that unittests are only instantiated if the module they
> > occur in is being compiled, otherwise ignore them (even in the
> > face of -unittest).  This way, adding unittests to Phobos won't
> > cause unintentional slowdowns / unittest bloat across *all* D
> > projects that import the affected Phobos modules. (Seen from
> > this angle, it's a pretty hefty cost.)
>
> Would it help to take the approach of mir, i.e. putting
> version(mir_test) before all the unittests?

It would, but if we have to do that in front of all unittest blocks in a
library, I think that that's a strong sign that the current design has a
serious problem that should be fixed if possible.

- Jonathan M Davis



Re: Bye bye, fast compilation times

2018-02-06 Thread Jonathan M Davis via Digitalmars-d
On Tuesday, February 06, 2018 17:20:04 H. S. Teoh via Digitalmars-d wrote:
> So I'd like to propose that we do something similar to what we did with
> template instantiations a couple of years ago: make it so that unittests
> are only instantiated if the module they occur in is being compiled,
> otherwise ignore them (even in the face of -unittest).  This way, adding
> unittests to Phobos won't cause unintentional slowdowns / unittest bloat
> across *all* D projects that import the affected Phobos modules. (Seen
> from this angle, it's a pretty hefty cost.)

Well, that sounds like a DIP is in order.

> Of course, this statement has to be qualified a bit: it's probably still
> a good idea to instantiate unittests inside templates -- since
> ostensibly they could be sanity-checking specific instantiations of
> templates. Though I'd argue that 90% of the time, their location inside
> a template is a mistake; they really should be outside templates and
> only instantiated once, because generally they test specific
> instantiations of the template rather than implement generic tests that
> apply across all template instantiations.

I agree that most of the time, unittest blocks inside templates shouldn't be
compiled with the template and that that mistake is made _way_ too often
(even in Phobos), but once in a while, it's actually desirable. I created a
DIP previously for using static to indicate that a unittest block inside a
template should not be treated as part of the template, but I haven't moved
it from the wiki to github, and Andrei pretty much told me that he though
that such a DIP was a waste of time and that I shouldn't continue with it.
He seems to think that the solutions involving using static ifs on all of
the unittest blocks and compiling them in with a specific instantiation only
are perfectly reasonable and that it's not worth adding anything to the
language for it. I don't agree, but I have no idea how to convince him.

Regardless, because some unittest blocks really should be compiled in with
every instantiation of a template (even if it's rare), I think that it's
clear that unittest blocks that are instantiated with a template should be
included in a program so long as any module that is specifically compiled
instantiates the template. It may be that the implementation would require
that they be there regardless, but even if that's true, not including the
non-templated unittest blocks which are not explicitly compiled in would
definitely be an improvement.

- Jonathan M Davis



Re: Bye bye, fast compilation times

2018-02-06 Thread jmh530 via Digitalmars-d

On Wednesday, 7 February 2018 at 01:20:04 UTC, H. S. Teoh wrote:
So I'd like to propose that we do something similar to what we 
did with template instantiations a couple of years ago: make it 
so that unittests are only instantiated if the module they 
occur in is being compiled, otherwise ignore them (even in the 
face of -unittest).  This way, adding unittests to Phobos won't 
cause unintentional slowdowns / unittest bloat across *all* D 
projects that import the affected Phobos modules. (Seen from 
this angle, it's a pretty hefty cost.)




Would it help to take the approach of mir, i.e. putting 
version(mir_test) before all the unittests?


Re: Bye bye, fast compilation times

2018-02-06 Thread H. S. Teoh via Digitalmars-d
On Wed, Feb 07, 2018 at 01:22:02AM +, Andres Clari via Digitalmars-d wrote:
[...]
> Is there some tutorial or example for using SCons with dub
> dependencies?

Not that I know of.  Basically what I did was:

- Create a dummy dub project in a subdirectory, containing a dummy
  source file containing an empty main().

- Declare whatever dub dependencies you need in this dummy project.

- Run `dub build -v` inside this subdirectory to make dub fetch
  dependencies, build libraries, etc..

- Parse the output, esp. the last few lines that show which include
  paths, linker flags, and libraries are required to build the main
  program.

- Specify these include paths, linker flags, and libraries in your
  SConstruct file for building your real project.

- Build away.

- If you need to refresh dependencies, go into the dummy project and run
  `dub build --force` to rebuild all dependencies, then run scons in
  your real project.

Arguably, some/all of the above could be automated by SCons. Though the
whole point is to *not* run dub every single time you build, so I'd keep
them separate, or as a non-default build target that only triggers when
you explicitly want it to.

Also, none of this is specific to SCons; you could use whatever other
build system you wish with the above steps.


T

-- 
This sentence is false.


Re: Bye bye, fast compilation times

2018-02-06 Thread H. S. Teoh via Digitalmars-d
On Mon, Feb 05, 2018 at 01:27:57PM -0800, H. S. Teoh via Digitalmars-d wrote:
> One of my D projects for the past while has been taking unusually long
> times to compile.  This morning, I finally decided to sit down and
> figure out exactly why.
[...]

I don't want this thread to turn into ragging on std.regex, so here's
the next instalment of this saga.

After taking std.regex out of the equation, I found that there was still
something else that was bottlenecking my builds.  So I went for another
hunt, and discovered another guilty party: std.format.


import std.format;
void main() { format("a"); }


Here I found something interesting: compiling with -unittest takes over
0.8s. But compiling *without* -unittest takes only 0.4s.

The compilation times of std.format itself aside (it's pretty
heavyweight because of heavy template usage, and the recent compile-time
format string checking, while pretty cool functionally, also comes at
the price of heavier CTFE, i.e., slower compilation -- I'd like to leave
digging into std.format for later), it's a little disturbing that
compiling with -unittest *doubled* the compilation time.

This reinforces a concern I've been having recently, that is, library
unittests that get instantiated when user code is compiled with
-unittest.  I'll be the first to say that having built-in unittests in D
has revolutionized the way I write code -- it has dramatically increased
the average quality of my code. But its current simplicity comes at a
cost: if you're using external libraries (i.e., outside the immediate
codebase you're working with, so that includes Phobos), compiling with
-unittest automatically inherits all unittests from all libraries that
you import, even if said unittests have nothing to do with your own
code.

Put another way: why should you be concerned with Phobos unittests when
you're building your own code?  Phobos unittesting should have been (and
is) already done by the autotester and CIs on github; there's no reason
these tests have to be run over and over again in user code.

So I'd like to propose that we do something similar to what we did with
template instantiations a couple of years ago: make it so that unittests
are only instantiated if the module they occur in is being compiled,
otherwise ignore them (even in the face of -unittest).  This way, adding
unittests to Phobos won't cause unintentional slowdowns / unittest bloat
across *all* D projects that import the affected Phobos modules. (Seen
from this angle, it's a pretty hefty cost.)

Of course, this statement has to be qualified a bit: it's probably still
a good idea to instantiate unittests inside templates -- since
ostensibly they could be sanity-checking specific instantiations of
templates. Though I'd argue that 90% of the time, their location inside
a template is a mistake; they really should be outside templates and
only instantiated once, because generally they test specific
instantiations of the template rather than implement generic tests that
apply across all template instantiations.


T

-- 
Never step over a puddle, always step around it. Chances are that
whatever made it is still dripping.


Re: Bye bye, fast compilation times

2018-02-06 Thread Andres Clari via Digitalmars-d

On Wednesday, 7 February 2018 at 00:36:22 UTC, H. S. Teoh wrote:
On Tue, Feb 06, 2018 at 11:20:53PM +, Andres Clari via 
Digitalmars-d wrote: [...]

[...]


I seem to vaguely recall that in some cases, ctRegex might even 
perform slower than regex().  But either way, my use cases for 
regexes generally aren't performance-sensitive enough to be 
worth the trouble of huge compilation time slowdown -- I just 
use regex() instead of ctRegex.



[...]

[...]


That depends on what you're doing with it, and also how you're 
building it. 3500+ lines isn't a lot of code; it ought to 
compile pretty fast unless you're using a lot of (1) templates, 
(2) CTFE.  Also, I find that dub builds are excruciatingly slow 
compared to just invoking dmd directly, due to network access 
and rescanning dependencies on every invocation.  I have a 
4700+ line vibe.d project; Diet templates are 
template/CTFE-heavy and generally take the longest to build. (I 
dumped dub and went back to an SCons-based system with separate 
compilation for major subsystems -- as long as I don't 
recompile Diet templates, the whole thing can build within 
seconds; with Diet templates it takes about 30 seconds :-/.)



T


Well I'm using vibe.d, but not templates on this project, just a 
minimal rest service, and a few timers and runTasks. So yeah I 
don't see why it should slow down that much.


Is there some tutorial or example for using SCons with dub 
dependencies?


Re: Bye bye, fast compilation times

2018-02-06 Thread H. S. Teoh via Digitalmars-d
On Tue, Feb 06, 2018 at 11:20:53PM +, Andres Clari via Digitalmars-d wrote:
[...]
> Switching from ctRegex! to regex yielded a 50% build time reduction,
> and from what I read even the normal regex are slowing things down
> considerably.

I seem to vaguely recall that in some cases, ctRegex might even perform
slower than regex().  But either way, my use cases for regexes generally
aren't performance-sensitive enough to be worth the trouble of huge
compilation time slowdown -- I just use regex() instead of ctRegex.


[...]
> Btw, my project which is 3517 lines of D builds in 20s disabling the
> ctRegex on an i7 4770k at 4.3Ghz.
> 
> So I'd say once you start doing some more complex usages, D's build
> speed goes out the door.

That depends on what you're doing with it, and also how you're building
it. 3500+ lines isn't a lot of code; it ought to compile pretty fast
unless you're using a lot of (1) templates, (2) CTFE.  Also, I find that
dub builds are excruciatingly slow compared to just invoking dmd
directly, due to network access and rescanning dependencies on every
invocation.  I have a 4700+ line vibe.d project; Diet templates are
template/CTFE-heavy and generally take the longest to build. (I dumped
dub and went back to an SCons-based system with separate compilation for
major subsystems -- as long as I don't recompile Diet templates, the
whole thing can build within seconds; with Diet templates it takes about
30 seconds :-/.)


T

-- 
Unix was not designed to stop people from doing stupid things, because that 
would also stop them from doing clever things. -- Doug Gwyn


Re: Bye bye, fast compilation times

2018-02-06 Thread Nick Sabalausky (Abscissa) via Digitalmars-d

On 02/06/2018 01:11 AM, Dmitry Olshansky wrote:

On Tuesday, 6 February 2018 at 05:45:35 UTC, Steven Schveighoffer wrote:

On 2/6/18 12:35 AM, Dmitry Olshansky wrote:


That’s really bad idea - isEmail is template so the burden of 
freaking slow ctRegex
is paid on per instantiation basis. Could be horrible with separate 
compilation.


Obviously it is horrible. On my mac, it took about 2.5 seconds to 
compile this one line.


I'm not sure how to fix it though... I suppose you could make


Just use the run-time version, it’s not that much slower. But then again 
static ipRegex = regex(...) will parse and build regex at CTFE.


Maybe lazy init?



If the regex string isn't dependent on the template's params, just move 
the regex outside the template.




Re: Bye bye, fast compilation times

2018-02-06 Thread Andres Clari via Digitalmars-d
On Tuesday, 6 February 2018 at 22:51:51 UTC, Steven Schveighoffer 
wrote:

On 2/6/18 5:23 PM, Walter Bright wrote:

On 2/6/2018 2:03 PM, Jacob Carlborg wrote:

On 2018-02-06 21:11, Walter Bright wrote:

std.string.isEmail() in D1 was a simple function. Maybe 
regex is just the wrong solution for this problem.


If I recall correctly, the current implementation of 
std.net.isEmail was requested by you.


Regardless of whether it was requested by me or not, if the 
current version is not working for us, we need to explore 
alternatives.


The regex problem is being solved:

https://github.com/dlang/phobos/pull/6129

-Steve


That's fixing just the "isEmail" issue which is good I guess.

But after reading this thread, I run some tests on one of my code 
bases, which uses about 6 regex throughout.


Switching from ctRegex! to regex yielded a 50% build time 
reduction, and from what I read even the normal regex are slowing 
things down considerably.


Might need a warning on the docs for ctRegex! explaining it'll 
screw your build times if you use it, unless there's some way to 
speed that up to something normal.


Btw, my project which is 3517 lines of D builds in 20s disabling 
the ctRegex on an i7 4770k at 4.3Ghz.


So I'd say once you start doing some more complex usages, D's 
build speed goes out the door.


Re: Bye bye, fast compilation times

2018-02-06 Thread Steven Schveighoffer via Digitalmars-d

On 2/6/18 5:23 PM, Walter Bright wrote:

On 2/6/2018 2:03 PM, Jacob Carlborg wrote:

On 2018-02-06 21:11, Walter Bright wrote:

std.string.isEmail() in D1 was a simple function. Maybe regex is just 
the wrong solution for this problem.


If I recall correctly, the current implementation of std.net.isEmail 
was requested by you.


Regardless of whether it was requested by me or not, if the current 
version is not working for us, we need to explore alternatives.


The regex problem is being solved:

https://github.com/dlang/phobos/pull/6129

-Steve


Re: Bye bye, fast compilation times

2018-02-06 Thread H. S. Teoh via Digitalmars-d
On Tue, Feb 06, 2018 at 02:29:07PM -0800, Walter Bright via Digitalmars-d wrote:
> On 2/6/2018 12:30 PM, Steven Schveighoffer wrote:
> > The regex in question I think is to ensure an email address like
> > abc@192.168.0.5 has a valid IP address. The D1 function doesn't
> > support that requirement.
> > 
> > I admit, I've never used it, so I don't know why it needs to be so
> > complex. But I assume some people depend on that functionality.
> 
> Regex is well known to not always be the best solution for string
> processing tasks. For example, it does not work well at all where
> recursion is desired, and nobody uses regex for lexer in a compiler.

Are you sure?  What about lex and its successors, like flex?

Of course, one could argue that the generated code isn't strictly a
regex implementation in the same way as std.regex... but isn't that just
a QoI issue? 


T

-- 
Life would be easier if I had the source code. -- YHL


Re: Bye bye, fast compilation times

2018-02-06 Thread Walter Bright via Digitalmars-d

On 2/6/2018 12:30 PM, Steven Schveighoffer wrote:
The regex in question I think is to ensure an email address like abc@192.168.0.5 
has a valid IP address. The D1 function doesn't support that requirement.


I admit, I've never used it, so I don't know why it needs to be so complex. But 
I assume some people depend on that functionality.


Regex is well known to not always be the best solution for string processing 
tasks. For example, it does not work well at all where recursion is desired, and 
nobody uses regex for lexer in a compiler.


Re: Bye bye, fast compilation times

2018-02-06 Thread Walter Bright via Digitalmars-d

On 2/6/2018 2:03 PM, Jacob Carlborg wrote:

On 2018-02-06 21:11, Walter Bright wrote:

std.string.isEmail() in D1 was a simple function. Maybe regex is just the 
wrong solution for this problem.


If I recall correctly, the current implementation of std.net.isEmail was 
requested by you.


Regardless of whether it was requested by me or not, if the current version is 
not working for us, we need to explore alternatives.


Re: Bye bye, fast compilation times

2018-02-06 Thread Jacob Carlborg via Digitalmars-d

On 2018-02-06 21:11, Walter Bright wrote:

std.string.isEmail() in D1 was a simple function. Maybe regex is just 
the wrong solution for this problem.


If I recall correctly, the current implementation of std.net.isEmail was 
requested by you.


--
/Jacob Carlborg


Re: Bye bye, fast compilation times

2018-02-06 Thread Timothee Cour via Digitalmars-d
another weird gotcha:
  auto s="foo".isEmail;
  writeln(s.toString); // ok
  writeln(s); // compile error


On Tue, Feb 6, 2018 at 12:30 PM, Steven Schveighoffer via
Digitalmars-d  wrote:
> On 2/6/18 3:11 PM, Walter Bright wrote:
>>
>> On 2/5/2018 9:35 PM, Dmitry Olshansky wrote:
>>>
>>> That’s really bad idea - isEmail is template so the burden of freaking
>>> slow ctRegex
>>> is paid on per instantiation basis. Could be horrible with separate
>>> compilation.
>>
>>
>> std.string.isEmail() in D1 was a simple function. Maybe regex is just the
>> wrong solution for this problem.
>>
>
> The regex in question I think is to ensure an email address like
> abc@192.168.0.5 has a valid IP address. The D1 function doesn't support that
> requirement.
>
> I admit, I've never used it, so I don't know why it needs to be so complex.
> But I assume some people depend on that functionality.
>
> -Steve



Re: Bye bye, fast compilation times

2018-02-06 Thread Steven Schveighoffer via Digitalmars-d

On 2/6/18 3:11 PM, Walter Bright wrote:

On 2/5/2018 9:35 PM, Dmitry Olshansky wrote:
That’s really bad idea - isEmail is template so the burden of freaking 
slow ctRegex
is paid on per instantiation basis. Could be horrible with separate 
compilation.


std.string.isEmail() in D1 was a simple function. Maybe regex is just 
the wrong solution for this problem.




The regex in question I think is to ensure an email address like 
abc@192.168.0.5 has a valid IP address. The D1 function doesn't support 
that requirement.


I admit, I've never used it, so I don't know why it needs to be so 
complex. But I assume some people depend on that functionality.


-Steve


Re: Bye bye, fast compilation times

2018-02-06 Thread Walter Bright via Digitalmars-d

On 2/5/2018 9:35 PM, Dmitry Olshansky wrote:

That’s really bad idea - isEmail is template so the burden of freaking slow 
ctRegex
is paid on per instantiation basis. Could be horrible with separate compilation.


std.string.isEmail() in D1 was a simple function. Maybe regex is just the wrong 
solution for this problem.


-- std.string.isEmail 

/***
 * Does string s[] start with an email address?
 * Returns:
 *  nullit does not
 *  char[]  it does, and this is the slice of s[] that is that email address
 * References:
 *  RFC2822
 */
char[] isEmail(char[] s)
{   size_t i;

if (!isalpha(s[0]))
goto Lno;

for (i = 1; 1; i++)
{
if (i == s.length)
goto Lno;
auto c = s[i];
if (isalnum(c))
continue;
if (c == '-' || c == '_' || c == '.')
continue;
if (c != '@')
goto Lno;
i++;
break;
}
//writefln("test1 '%s'", s[0 .. i]);

/* Now do the part past the '@'
 */
size_t lastdot;
for (; i < s.length; i++)
{
auto c = s[i];
if (isalnum(c))
continue;
if (c == '-' || c == '_')
continue;
if (c == '.')
{
lastdot = i;
continue;
}
break;
}
if (!lastdot || (i - lastdot != 3 && i - lastdot != 4))
goto Lno;

return s[0 .. i];

Lno:
return null;
}


Re: Bye bye, fast compilation times

2018-02-06 Thread Steven Schveighoffer via Digitalmars-d

On 2/6/18 2:07 PM, H. S. Teoh wrote:

I'm not sure I'm seeing the value of using ctRegex here.  What's wrong
with a module static runtime regex initialized by a static this()?


No, I'd rather have it initialized on first call.



And before anyone complains about initializing the regex if user code
never actually uses it, it's possible to use static this() on an
as-needed basis:

template ipRegex()
{
// Eponymous templates FTW!
Regex!char ipRegex;

static this()
{
ipRegex = regex(`blah blah blah`);
}
}

auto isEmail(... blah blah ...)
{
...
if (ipRegex.match(...)) ...
...
}

Basically, if `ipRegex` is never referenced, the template is never
instantiated and the static this() basically doesn't exist. :-D
Pay-as-you-go FTW!


You may not realize that this actually compiles it for ALL modules that 
use it, and the compiler puts in a gate to prevent it from running more 
than once. So you pay every time anyways (compile-time wise at least). 
It also makes any importing module now a module that defines a static 
ctor, so cycles are much more likely.


In any case, there is a PR in the works that should eliminate the need 
for regex altogether: https://github.com/dlang/phobos/pull/6129


-Steve


Re: Bye bye, fast compilation times

2018-02-06 Thread H. S. Teoh via Digitalmars-d
On Tue, Feb 06, 2018 at 05:35:44AM +, Dmitry Olshansky via Digitalmars-d 
wrote:
> On Tuesday, 6 February 2018 at 04:35:42 UTC, Steven Schveighoffer wrote:
> > On 2/5/18 11:09 PM, psychoticRabbit wrote:
[...]
> > > 
> > > import std.net.isemail;
> > > 
> > > void main()
> > > {
> > >      auto checkEmail = "some...@somewhere.com".isEmail();
> > > }
> > > 
> > 
> > I was surprised at this, then I looked at the first line of isEmail:
> > 
> > static ipRegex =
> > ctRegex!(`\b(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}`~
> > 
> > `(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)$`.to!(const(Char)[]));
> > 
> > So it's really still related to regex.

Yeah, ctRegex is a bear at compile-time.  Why can't we just use a
runtime regex?  It will at least take "only" 3 seconds to compile. :-D
Or just don't use a regex at all.


> That’s really bad idea - isEmail is template so the burden of freaking
> slow ctRegex is paid on per instantiation basis. Could be horrible
> with separate compilation.
[...]

I'm not sure I'm seeing the value of using ctRegex here.  What's wrong
with a module static runtime regex initialized by a static this()?

And before anyone complains about initializing the regex if user code
never actually uses it, it's possible to use static this() on an
as-needed basis:

template ipRegex()
{
// Eponymous templates FTW!
Regex!char ipRegex;

static this()
{
ipRegex = regex(`blah blah blah`);
}
}

auto isEmail(... blah blah ...)
{
...
if (ipRegex.match(...)) ...
...
}

Basically, if `ipRegex` is never referenced, the template is never
instantiated and the static this() basically doesn't exist. :-D
Pay-as-you-go FTW!


T

-- 
If you want to solve a problem, you need to address its root cause, not just 
its symptoms. Otherwise it's like treating cancer with Tylenol...


Re: Bye bye, fast compilation times

2018-02-06 Thread H. S. Teoh via Digitalmars-d
On Tue, Feb 06, 2018 at 05:44:17AM +, Dmitry Olshansky via Digitalmars-d 
wrote:
[...]
> Honestly I’m tired to hell of working with our compiler and its
> compile time features. When it doesn’t pee itself due to OOM I’m
> almost happy.

Heh, dmd's famous memory usage is causing me tons of grief on low-memory
systems, too.  Basically if you have anything less than 2GB of RAM,
you might as well give up trying to compile anything non-trivial. We
need to get a serious handle on dmd's memory consumption -- at least let
there be an option or something that will turn out the GC or whatever.
It's better for dmd to be (gosh) slow, than for it not to be able to
compile anything at all due to it provoking the kernel OOM killer.


> In retrospect I should have just provided a C interface and compiled
> the whole thing separately. And CTFE could easily be replaced by a
> small custom JIT compiler, it would also work at run-time(!).

We seriously need to get newCTFE finished and merged.  Stefan is very
busy with other stuff ATM; I wonder if a few of us can continue his work
and get newCTFE into a mergeable state.  Given how much D's
"compile-time" features are advertised, and D's new (ick) slogan of
being fast or whatever, it's high time we actually delivered on our
promises by actually making CTFE more usable.

On that note, though, I think a JIT regex compiler totally makes sense.
I'd totally support that.


> Especially considering that it’s been 6 years but it’s still is not
> practical to use ctRegex.

I find that using just plain `regex` is Good Enough(tm) for my purposes.
Do we really need ctRegex?  The idea of generating an optimal FSM at
compile-time is rather appealing, but in the grand scheme of things,
doesn't seem like an absolute must-have.


> > The latter department as also suffered a regression; see for
> > example: https://github.com/dlang/phobos/pull/5981.)
> 
> Yup, Martin seems on top of it, thankfully.
[...]

Unfortunately, Martin's PR is only to improve runtime performance.  It's
still dog-slow to *compile* std.regex. :-(


T

-- 
Dogs have owners ... cats have staff. -- Krista Casada


Re: Bye bye, fast compilation times

2018-02-06 Thread Dmitry Olshansky via Digitalmars-d

On Tuesday, 6 February 2018 at 13:51:01 UTC, Nathan S. wrote:
Just use the run-time version, it’s not that much slower. But 
then again static ipRegex = regex(...) will parse and build 
regex at CTFE.


Maybe lazy init?


FYI I've made a pull request that replaces uses of regexes in 
std.net.isemail. It turns out they weren't being used for 
anything indispensable. Import benchmark results were 
encouraging.


https://github.com/dlang/phobos/pull/6129


Then again if you may not need regex for IPv4 / IPv6.

In theory it should have been the goto case for ctRegex but not 
at the cost of such horrible compile times.




Re: Bye bye, fast compilation times

2018-02-06 Thread Nathan S. via Digitalmars-d
On Tuesday, 6 February 2018 at 06:11:55 UTC, Dmitry Olshansky 
wrote:
On Tuesday, 6 February 2018 at 05:45:35 UTC, Steven 
Schveighoffer wrote:

On 2/6/18 12:35 AM, Dmitry Olshansky wrote:


That’s really bad idea - isEmail is template so the burden of 
freaking slow ctRegex
is paid on per instantiation basis. Could be horrible with 
separate compilation.


Obviously it is horrible. On my mac, it took about 2.5 seconds 
to compile this one line.


I'm not sure how to fix it though... I suppose you could make


Just use the run-time version, it’s not that much slower. But 
then again static ipRegex = regex(...) will parse and build 
regex at CTFE.


Maybe lazy init?


FYI I've made a pull request that replaces uses of regexes in 
std.net.isemail. It turns out they weren't being used for 
anything indispensable. Import benchmark results were encouraging.


https://github.com/dlang/phobos/pull/6129


Re: Bye bye, fast compilation times

2018-02-05 Thread Dmitry Olshansky via Digitalmars-d
On Tuesday, 6 February 2018 at 05:45:35 UTC, Steven Schveighoffer 
wrote:

On 2/6/18 12:35 AM, Dmitry Olshansky wrote:


That’s really bad idea - isEmail is template so the burden of 
freaking slow ctRegex
is paid on per instantiation basis. Could be horrible with 
separate compilation.


Obviously it is horrible. On my mac, it took about 2.5 seconds 
to compile this one line.


I'm not sure how to fix it though... I suppose you could make


Just use the run-time version, it’s not that much slower. But 
then again static ipRegex = regex(...) will parse and build regex 
at CTFE.


Maybe lazy init?

it 3 overloads, but this defeats a lot of the purpose of having 
templates in the first place.


-Steve





Re: Bye bye, fast compilation times

2018-02-05 Thread Steven Schveighoffer via Digitalmars-d

On 2/6/18 12:35 AM, Dmitry Olshansky wrote:

On Tuesday, 6 February 2018 at 04:35:42 UTC, Steven Schveighoffer wrote:

On 2/5/18 11:09 PM, psychoticRabbit wrote:

On Monday, 5 February 2018 at 21:27:57 UTC, H. S. Teoh wrote:


Comment out the call to `regex()`, and I get:

--
real    0m0.285s
user    0m0.262s
sys 0m0.023s
--



regex is not the only one I avoid..

how long you think this takes to compile?
(try ldc2 too ..just for laughs ;-)


import std.net.isemail;

void main()
{
 auto checkEmail = "some...@somewhere.com".isEmail();
}



I was surprised at this, then I looked at the first line of isEmail:

    static ipRegex = 
ctRegex!(`\b(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}`~


`(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)$`.to!(const(Char)[]));

So it's really still related to regex.



That’s really bad idea - isEmail is template so the burden of freaking 
slow ctRegex
is paid on per instantiation basis. Could be horrible with separate 
compilation.


Obviously it is horrible. On my mac, it took about 2.5 seconds to 
compile this one line.


I'm not sure how to fix it though... I suppose you could make it 3 
overloads, but this defeats a lot of the purpose of having templates in 
the first place.


-Steve


Re: Bye bye, fast compilation times

2018-02-05 Thread Dmitry Olshansky via Digitalmars-d

On Monday, 5 February 2018 at 21:27:57 UTC, H. S. Teoh wrote:
One of my D projects for the past while has been taking 
unusually long times to compile.  This morning, I finally 
decided to sit down and figure out exactly why. What I found 
was rather disturbing:


--
import std.regex;
void main() {
auto re = regex(``);
}
--

Compile command: time dmd -c test.d

Output:
--
real0m3.113s
user0m2.884s
sys 0m0.226s
--

Comment out the call to `regex()`, and I get:

--
real0m0.285s
user0m0.262s
sys 0m0.023s
--



Clearly, something is wrong if the mere act of compiling a 
regex causes a 4-line program to take *3 seconds* to compile,


There is a fuckton of templates involved, plus a couple of tries 
are built at CTFE.
The regression is curious though, maybe something gets recomputed 
at CTFE over and over again.



where normally dmd takes less than a second.


Honestly I’m tired to hell of working with our compiler and its 
compile time features. When it doesn’t pee itself due to OOM I’m 
almost happy.


In retrospect I should have just provided a C interface and 
compiled the whole thing separately. And CTFE could easily be 
replaced by a small custom JIT compiler, it would also work at 
run-time(!).


Especially considering that it’s been 6 years but it’s still is 
not practical to use ctRegex.


The latter department as also suffered a regression; see for 
example: https://github.com/dlang/phobos/pull/5981.)




Yup, Martin seems on top of it, thankfully.



T





Re: Bye bye, fast compilation times

2018-02-05 Thread Dmitry Olshansky via Digitalmars-d
On Tuesday, 6 February 2018 at 04:35:42 UTC, Steven Schveighoffer 
wrote:

On 2/5/18 11:09 PM, psychoticRabbit wrote:

On Monday, 5 February 2018 at 21:27:57 UTC, H. S. Teoh wrote:


Comment out the call to `regex()`, and I get:

--
real    0m0.285s
user    0m0.262s
sys 0m0.023s
--



regex is not the only one I avoid..

how long you think this takes to compile?
(try ldc2 too ..just for laughs ;-)


import std.net.isemail;

void main()
{
     auto checkEmail = "some...@somewhere.com".isEmail();
}



I was surprised at this, then I looked at the first line of 
isEmail:


static ipRegex = 
ctRegex!(`\b(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}`~


`(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)$`.to!(const(Char)[]));

So it's really still related to regex.



That’s really bad idea - isEmail is template so the burden of 
freaking slow ctRegex
is paid on per instantiation basis. Could be horrible with 
separate compilation.



-Steve





Re: Bye bye, fast compilation times

2018-02-05 Thread rikki cattermole via Digitalmars-d

On 06/02/2018 4:35 AM, Steven Schveighoffer wrote:

On 2/5/18 11:09 PM, psychoticRabbit wrote:

On Monday, 5 February 2018 at 21:27:57 UTC, H. S. Teoh wrote:


Comment out the call to `regex()`, and I get:

--
real    0m0.285s
user    0m0.262s
sys 0m0.023s
--



regex is not the only one I avoid..

how long you think this takes to compile?
(try ldc2 too ..just for laughs ;-)


import std.net.isemail;

void main()
{
 auto checkEmail = "some...@somewhere.com".isEmail();
}



I was surprised at this, then I looked at the first line of isEmail:

     static ipRegex = 
ctRegex!(`\b(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}`~


`(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)$`.to!(const(Char)[]));

So it's really still related to regex.

-Steve


On that note, we really should remove it performance-aside, you cannot 
really trust it.


Re: Bye bye, fast compilation times

2018-02-05 Thread Steven Schveighoffer via Digitalmars-d

On 2/5/18 11:09 PM, psychoticRabbit wrote:

On Monday, 5 February 2018 at 21:27:57 UTC, H. S. Teoh wrote:


Comment out the call to `regex()`, and I get:

--
real    0m0.285s
user    0m0.262s
sys 0m0.023s
--



regex is not the only one I avoid..

how long you think this takes to compile?
(try ldc2 too ..just for laughs ;-)


import std.net.isemail;

void main()
{
     auto checkEmail = "some...@somewhere.com".isEmail();
}



I was surprised at this, then I looked at the first line of isEmail:

static ipRegex = 
ctRegex!(`\b(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}`~


`(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)$`.to!(const(Char)[]));

So it's really still related to regex.

-Steve


Re: Bye bye, fast compilation times

2018-02-05 Thread psychoticRabbit via Digitalmars-d
On Tuesday, 6 February 2018 at 04:09:24 UTC, psychoticRabbit 
wrote:

how long you think this takes to compile?
(try ldc2 too ..just for laughs ;-)


import std.net.isemail;

void main()
{
auto checkEmail = "some...@somewhere.com".isEmail();
}



oh.. and for an even bigger laugh... -O -release  (ldc2 took ~10 
seconds)





Re: Bye bye, fast compilation times

2018-02-05 Thread psychoticRabbit via Digitalmars-d

On Monday, 5 February 2018 at 21:27:57 UTC, H. S. Teoh wrote:


Comment out the call to `regex()`, and I get:

--
real0m0.285s
user0m0.262s
sys 0m0.023s
--



regex is not the only one I avoid..

how long you think this takes to compile?
(try ldc2 too ..just for laughs ;-)


import std.net.isemail;

void main()
{
auto checkEmail = "some...@somewhere.com".isEmail();
}