date:20140504

On Sat, 2014-05-03 at 19:37 +, Atila Neves via Digitalmars-d wrote:
[…]
 I'm using parallel and taskPool from std.parallelism. I was under 
 the impression it gave me a ready-to-use pool with as many 
 threads as I have cores.

There is a default, related to the number of cores the OS thinks there
is (*), but you can also set the number manually.  std.parallelism could
do with some work to make it better than it already is.


(*) Physical cores are not necessarily the number reported by the OS due
to core hyperthreads. Quad core no hyperthreads, and dual core, two
hyperthreads per core, both get reported as four processor systems.
However if you benchmark them you get very, very different performance
characteristics.

-- 
Russel.
=
Dr Russel Winder  t: +44 20 7585 2200   voip: sip:russel.win...@ekiga.net
41 Buckmaster Roadm: +44 7770 465 077   xmpp: rus...@winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

Re: Progress on Adam Wilson's Graphics API?

2014-05-04 Thread Joseph Rushton Wakeling via Digitalmars-d

On Sat, 2014-05-03 at 22:34 +, Nordlöw via Digitalmars-d wrote:
  You can follow all progress on GitHub here: 
  https://github.com/auroragraphics/
 
 Ok. Now I know :)

The repository is reported as being empty :-(

-- 
Russel.
=
Dr Russel Winder  t: +44 20 7585 2200   voip: sip:russel.win...@ekiga.net
41 Buckmaster Roadm: +44 7770 465 077   xmpp: rus...@winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

Re: isUniformRNG


On 04/05/14 04:03, Nick Sabalausky via Digitalmars-d wrote:

In std.random, is the isUniformRNG intended to determine whether the given
type is *some* RNG or just a *specific* form of RNG? Because I don't see any
isRNG that's more general.


Yes, it is meant to denote that the type is a _uniform_ random number generator, 
that is, it's an input range whose elements are numbers drawn from a uniform 
distribution.  One could even be more strict as per the C++11 standard and 
specify that it's a range whose elements are unsigned integral types uniformly 
distributed in the closed interval [min, max].


Personally speaking, I have spent quite a lot of time being uncertain about 
whether the constraint to only unsigned integral types is entirely appropriate 
(e.g. Boost::Random contains RNGs whose underlying type is floating-point), but 
it _does_ greatly simplify many design factors.  For example, the fact that 
you're given a guarantee about bounds makes things much simpler when you move on 
to (say) random distributions, such as the uniform distribution, where you want 
to control precisely the upper and lower bound behaviour (closed or open).


About a more general isRNG template: can you be more precise about what you 
are interested in achieving with this?  Generally speaking I would find it 
rather dangerous to go passing around sources of randomness without having some 
understanding of their properties :-)


I should also add: the C++11 standard distinguishes between uniform random 
number generators (the aforementioned 
uniformly-distributed-unsigned-integers-in-[min, max]) and uniform 
_distributions_, which take the output of a uniform RNG and transform it into 
random values drawn from other distributions.  Again, here we can see the 
importance of C++11's strict definition of a uniform RNG: suppose we take a 
uniform _distribution_ of floating-point numbers in the half-open range [0, 1) 
and use it as the _source_ of randomness for a uniform distribution in the 
closed interval [0, 1].  It won't work, because the assumptions about bounds 
won't work.



More importantly, should a crypto RNG count as isUniformRNG?


If it's producing unsigned integral types uniformly distributed in a closed 
interval [min, max], why not?

Re: std.allocator: false pointers


On Saturday, 3 May 2014 at 23:41:02 UTC, safety0ff wrote:
Well, in a 64-bit address space, the false pointer issue is 
almost
mute, the issue comes in when you try to apply this design to 
32-bit,


That assumes that the heap is located at a high address.

I think false pointers must be addressed to make claims that D 
is well-behaved on 32-bit systems.


Which means a precise collector, which means faster stack meta 
info access, which means faster exception handling? Then use a 
single pass mark-and-don't-sweep collector/allocator? (invert the 
meaning of the bit for every other full collection cycle)


A conservative collector will never be acceptable in a language 
which is primarily GC based. It is something you can bolt on to a 
RC system to collrct cycles, IMHO.

Scenario: OpenSSL in D language, pros/cons

2014-05-04 Thread Daniele M. via Digitalmars-d


I have read this excellent article by David A. Wheeler:

http://www.dwheeler.com/essays/heartbleed.html

And since D language was not there, I mentioned it to him as a 
possible good candidate due to its static typing and related 
features.


However, now I am asking the community here: would a D 
implementation (with GC disabled) of OpenSSL have been free from 
Heartbleed-type vulnerabilities? Specifically 
http://cwe.mitre.org/data/definitions/126.html and 
http://cwe.mitre.org/data/definitions/20.html as David mentions.


I find this perspective very interesting, please advise :)

Re: Running Phobos unit tests in threads: I have data

2014-05-04 Thread Atila Neves via Digitalmars-d

On Saturday, 3 May 2014 at 22:46:03 UTC, Andrei Alexandrescu 
wrote:

On 5/3/14, 2:42 PM, Atila Neves wrote:
gdc gave _very_ different results. I had to use different 
modules
because at some point tests started failing, but with gdc the 
threaded

version runs ~3x faster.

On my own unit-threaded benchmarks, running the UTs for 
Cerealed over
and over again was only slightly slower with threads than 
without. With

dmd the threaded version was nearly 3x slower.


Sounds like a severe bug in dmd or dependents. -- Andrei


Seems like it. Just to be sure I swapped ld.gold for ld.bfd and 
the problem was still there.


I'm not entirely sure how to file this bug: with just my simple 
example above?


Atila

Re: Running Phobos unit tests in threads: I have data

2014-05-04 Thread Atila Neves via Digitalmars-d

Like I mentioned afterwards, I tried a different number of 
threads. On my machine, at least, std.parallelism.totalCPUs 
returns 8, the number of virtual cores. As it should.


Atila

On Sunday, 4 May 2014 at 07:49:51 UTC, Russel Winder via 
Digitalmars-d wrote:
On Sat, 2014-05-03 at 19:37 +, Atila Neves via 
Digitalmars-d wrote:

[…]
I'm using parallel and taskPool from std.parallelism. I was 
under the impression it gave me a ready-to-use pool with as 
many threads as I have cores.


There is a default, related to the number of cores the OS 
thinks there
is (*), but you can also set the number manually.  
std.parallelism could

do with some work to make it better than it already is.


(*) Physical cores are not necessarily the number reported by 
the OS due
to core hyperthreads. Quad core no hyperthreads, and dual core, 
two
hyperthreads per core, both get reported as four processor 
systems.
However if you benchmark them you get very, very different 
performance

characteristics.

Re: More radical ideas about gc and reference counting


On Saturday, 3 May 2014 at 11:12:56 UTC, Michel Fortin wrote:
On 2014-05-01 17:35:36 +, Marc Schütz schue...@gmx.net 
said:


Maybe the language should have some way to distinguish between 
GC-managed and manually-managed objects, preferably in the 
type system. Then it could be statically checked whether an 
object is supposed to be GC-managed, and consequentially 
shouldn't have a destructor.


Or turn the rule on its head: make it so having a destructor 
makes the heap memory block reference counted. With this adding 
a destructor always cause deterministic destruction.


The compiler knows statically whether a struct has a 
destructor. For a class you need a runtime trick because the 
root object which can be either. Use a virtual call or a magic 
value in the reference count field to handle the reference 
count management. You also need a way to tag a class to be 
guarantied it has no derived class with a destructor (to 
provide a static proof for the compiler it can omit ARC code), 
perhaps @disable ~this().


Then remains the problem of cycles. It could be a hard error if 
the destructor is @safe (error thrown when the GC collects it). 
The destructor could be allowed to run (in any thread) if the 
destructor is @system or @trusted.


The interesting thing with this is that the current D semantics 
are preserved, destructors become deterministic (except in the 
presence of cycles, which the GC will detect for you), and if 
you're manipulating pointers to pure memory (memory blocks 
having no destructor) there's no ARC overhead. And finally, no 
new pointer attributes; Walter will like this last one.


This is certainly also an interesting idea, but I suspect it is 
bound to fail, simply because it involves ARC. Reference counting 
always makes things so much more complicated... See for example 
the cycles problem you mentioned: If you need a GC for that, you 
cannot guarantee that the objects will be collected, which was 
the reason to introduce ARC in the first place. Then there are 
the problems with shared vs. thread-local RC (including casting 
between the two), and arrays/slices of RC objects. And, of 
course, Walter doesn't like it ;-)

Re: [OT] Slides from 2014 International Symposium on Code Generation and Optimization


Paulo Pinto:


http://cgo.org/cgo2014/conference/program/


Very nice, thank you. Will read.

Bye,
bearophile

Re: The Current Status of DQt


A meta question, not related to your specific implementation:

While Qt is certainly the most powerful and comprehensive 
portable GUI framework, it also contains lots of code that's not 
related to user interfaces: strings, multi-threading support, 
file abstractions, containers, databases interfaces, etc. In 
short: Things which properly should be part of the standard 
library, but aren't (widely) available, or not working well 
enough in C++.


Now, Qt depends on those, and if you want to interact with it, 
you need to use them in your own code. Doesn't that cause a lot 
of friction? Just as a random example, QListView can take its 
elements from a data source (model). But in D, that data source 
might be idiomatically implemented as a range, so it has to be 
adapted first. Or, a more frequent thing: char[] vs. QString.


Could it be a better strategy to only re-use low-level 
functionality of Qt internally, but provide more idiomatic public 
D interfaces for it?

Re: D For A Web Developer

2014-05-04 Thread Danny Weldon via Digitalmars-d

On Wednesday, 30 April 2014 at 04:22:52 UTC, Rikki Cattermole 
wrote:
It may be a good time to repeat, we need a marketing manager 
for D!

Somebody really really needs to focus on getting us out there.


If somebody has some time, they could post a solution in D to 
this problem:


http://codegolf.stackexchange.com/questions/26323/how-slow-is-python-really-or-how-fast-is-your-language

It all helps to get the language visible.

Re: D For A Web Developer


Danny Weldon:

If somebody has some time, they could post a solution in D to 
this problem:


http://codegolf.stackexchange.com/questions/26323/how-slow-is-python-really-or-how-fast-is-your-language

It all helps to get the language visible.


Here are my solutions:
http://forum.dlang.org/thread/pvojsrqmaksqwokue...@forum.dlang.org

I don't have a Stackexchange account. You are welcome to post the 
two solutions (of two different problems).


Bye,
bearophile

Re: Running Phobos unit tests in threads: I have data

On Sun, 2014-05-04 at 08:47 +, Atila Neves via Digitalmars-d wrote:
 Like I mentioned afterwards, I tried a different number of 
 threads. On my machine, at least, std.parallelism.totalCPUs 
 returns 8, the number of virtual cores. As it should.

If you can create a small example of the problem, and I can remember how
to run std.parallelism as a separate module, I can try and take a look
at this later next week.

-- 
Russel.
=
Dr Russel Winder  t: +44 20 7585 2200   voip: sip:russel.win...@ekiga.net
41 Buckmaster Roadm: +44 7770 465 077   xmpp: rus...@winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

Re: Scenario: OpenSSL in D language, pros/cons

2014-05-04 Thread Jonathan M Davis via Digitalmars-d

On Sun, 04 May 2014 08:34:19 +
Daniele M. via Digitalmars-d digitalmars-d@puremagic.com wrote:

 I have read this excellent article by David A. Wheeler:

 http://www.dwheeler.com/essays/heartbleed.html

 And since D language was not there, I mentioned it to him as a
 possible good candidate due to its static typing and related
 features.

 However, now I am asking the community here: would a D
 implementation (with GC disabled) of OpenSSL have been free from
 Heartbleed-type vulnerabilities? Specifically
 http://cwe.mitre.org/data/definitions/126.html and
 http://cwe.mitre.org/data/definitions/20.html as David mentions.

 I find this perspective very interesting, please advise :)

@safe code protects against indexing an array out-of-bounds. So, if OpenSSL
had been implemented in D, and its heartbeat feature used @safe code, then
heartbleed would not have been possible. As soon as an attempt was made to
index passed the end of the array, it would have thrown a RangeError and
killed the program.

Now, even if OpenSSL had been implemented in D, if it had used @system or
@trusted code for its heartbeat feature, then it could have had the bug just
as easily in D as it did in C. And given all of the horrible practices in
OpenSSL, I very much doubt that having it written in D would have prevented
much, because anyone making the choices that the OpenSSL folks have been
making would likely have ended up with horrible D code which was mostly
@system and probably doing all kinds of things that are inherently risky,
forgoeing many of the benefits that D provides.

I think that it's safe to say that D makes it easier to catch problems like
this, but it doesn't entirely prevent them, and bad programming practices
can pretty much always get around protections that the language provides
unless the language provides no ways around those protections (which isn't
the case in D, because it's a systems language and needs to provide low-level
access and features for those programs that need them - it just doesn't use
those by default).

If I had more time, I'd actually be tempted to write an SSL implementation in
D, but even if I were to do an excellent job of it, it would still need to be
vetted by security experts to make sure that it didn't have horrible security
bugs in it (much as it would be likely that there would be fewer thanks to
the fact that it would be writen in D), and I suspect that it's the kind of
thing that many people aren't likely to trust because of how critical it is.
So, I don't know how good an idea it is at this point for someone to write
an implementation of SSL or TLS in D. Certainly, it's the type of thing where
we've generally tried to wrap existing C libraries in order to avoid having
to spend the time, effort, and expertise on in order to fully implement it
ourselves. The Go guys did it, but if I understand correctly, the fellow that
did it was one of the OpenSSL developers, so presumably he's already very
familiar with all of the ins and outs of SSL, and I don't know if any of us
here are (I'm certainly not - if I were doing it, I'd pretty much just have
to go off of its associated RFCs, for better or worse).

At this point though, if I were looking at using an existing implementation
of SSL, I'd probably be looking at GnuTLS rather than OpenSSL given how
horrible OpenSSL's codebase is. I don't know that GnuTLS is any better, but
it wouldn't be hard for it to be. OpenSSL is horrible both with regards to
its implementation and its API, and we'd all be better off if something
better replaced it (be it GnuTLS or something else). Unfortunately, even if
something better _were_ written in D, it's probably only the D folks who
would benefit, since it's not terribly likely at this point that very many
folks are going to wrap a D library in order to use it another language.

However, regardless of whether we ever end up with an SSL implementation in D,
I think that in the long run, D will show itself to be much, much better than
C or C++ at writing code that has a low number of security bugs.

- Jonathan M Davis

Re: opImplicitCast/opImplicitCastFrom

2014-05-04 Thread Nordlöw

When a programmer cares for integer overflow one can use 
Bounded!(T.min, T.max).


See mine extension of Adam's bounded.d:

https://github.com/nordlow/justd/blob/master/bound.d

I call it bound.d instead to save to characters :)

/Per

Re: More radical ideas about gc and reference counting

On Friday, 2 May 2014 at 21:12:12 UTC, H. S. Teoh via 
Digitalmars-d wrote:
On Fri, May 02, 2014 at 09:03:15PM +, monarch_dodra via 
Digitalmars-d wrote:
On Friday, 2 May 2014 at 15:06:59 UTC, Andrei Alexandrescu 
wrote:
So now it looks like dynamic arrays also can't contain 
structs with

destructors :o). -- Andrei

Well, that's always been the case, and even worst, since in a 
dynamic
array, destructor are guaranteed to *never* be run. 
Furthermore, given

the append causes relocation which duplicates, you are almost
*guaranteed* to leak your destructors. You just can't keep 
track of

the usage of a naked dynamic array.

This usually comes as a great surprise to users in .learn. 
It's also

the reason why using File[] never ends well...


This is why I'm unhappy with the way this is going. Current 
behaviour of
structs with dtors is already fragile enough, now we're pulling 
the rug
out from under classes as well. So that will be yet another 
case where

dtors won't work as expected.

I'm getting the feeling that dtors were bolted on as an 
afterthought,
and only works properly for a very narrow spectrum of use 
cases. Rather
than expand the usable cases, we're proposing to reduce them 
(by getting

rid of class dtors). I can't see *this* ending well either. :-(


Unfortunately, the rug they are standing is placed over a gaping 
hole in the ground, metaphorically speaking, so it never was a 
very stable situation to begin with. I'm not sure it makes sense 
to put much effort into increasing the likelihood that 
destructors are called, when we then _still_ can't guarantee it.


Better to acknowledge that it simply can't work (if that's the 
case), and try to find a way to avoid these situations. For 
example, we could statically forbid GC managed objects with 
destructors (not classes, but _anything_ GC managed). That means 
that calling new for these types, or allocating a dynamic array 
of them, will be an error. Of course, an escape hatch needs to be 
put in place. isolated might be a candidate.

Re: Progress on Adam Wilson's Graphics API?

2014-05-04 Thread Mike Parker via Digitalmars-d


On 5/4/2014 4:53 PM, Russel Winder via Digitalmars-d wrote:

On Sat, 2014-05-03 at 22:34 +, Nordlöw via Digitalmars-d wrote:

You can follow all progress on GitHub here:
https://github.com/auroragraphics/


Ok. Now I know :)


The repository is reported as being empty :-(



All of the work so far is going on in the DX11 repository.

Re: GC vs Resource management.


On 2014-05-03 14:28, monnoroch wrote:


That leaves only to determine, what objects are scoped. Well, that is
obviously stack-allocated structs, gc-allocated scope classes and
gc-allocated structs in scope classes.


Will the destructor of GC allocated scope classes be called when its 
surrounding scope exists?


--
/Jacob Carlborg

Re: D For A Web Developer


On 2014-05-03 21:52, Atila Neves wrote:


For me it's the output. I don't want to see the output of other
tests when I'm debugging a failure.


That's a good point.

--
/Jacob Carlborg

Re: D For A Web Developer


On 2014-05-03 21:51, Atila Neves wrote:


This is why I started to learn Cucumber.


Cucumber is for acceptance tests. There are also functional and 
integration tests.


--
/Jacob Carlborg

Re: GC vs Resource management.


On Saturday, 3 May 2014 at 12:28:03 UTC, monnoroch wrote:
Back to the dtors: i understand, that all the stuff you propose 
could make GC faster, simpler, and cooler,


Note that this is _not_ the motivation behind the discussed 
changes. It's primarily about correctness and consistency. What 
we currently have if something that doesn't work at all in some 
cases (dynamic arrays, new with structs), and where it does work 
(class destructors) users have to conform to various rules as to 
what is allowed in a GC-called destructor (e.g. don't access 
references to other objects, don't throw (?), don't use 
thread-local globals, any probably many more subtle rules), that 
mostly cannot be enforced statically or even at runtime, 
potentially causing silent corruption if you fail to comply with 
them. This is IMO more than enough motivation to do something 
about it.


but it sounds insane to anyone, who uses the language, not 
develops it, that if you create struct object, dror will be 
called, but if you place the copy in a container, it wont. It's 
just unanderstandable from user's point of view.


Not in a container, but making it GC managed. And IMO, it's 
unreasonable to expect deterministic behaviour from a tracing 
(non-precise) GC, which is inherently non-deterministic. The real 
problem is that you can currently do these things accidentally, 
without noticing it.



1) C.
That is the simplest way: fully-manual resource management.
It's obvious, we can't do that in D, because it's supposed to 
be simpler for coding, than C.


Again, the main motivation for GC is safety and enabling more 
idioms that would otherwise be impossible, _not_ lazyness on the 
part of the programmer.



3) C++.
This one is semi-automatic (talking about user code, not some 
allocator libraries): you choose the scheme (refcounting, 
unique reference) and then it'll do the rest.


4) Rust.
I'm not a pro here, but as i understand, it uses C++ way, and 
adds gc-collected pointers, but not sure, so help me here.


You could describe it like that, yes. But it's a lot more 
involved. In a way, Rust's owned types are the opposite of a GC: 
A GC destroys objects lazily and keeps them alive as long as they 
are referenced. Owned objects are destroyed eagerly at the end of 
their scope, but the type system ensures that there are no 
references to them at this point.



What i propose, is to include new concept in D: scoped objects.
Any object (no matter is it a class or struct instance) can be 
either scoped or not.
Dtors for scoped objects are called when out of scope, dtors 
for non-scoped objects are not called at all.


It is actually as simple as rewrite code

A a = A();

as

A a = A();
scope(exit)
a.~A();



There is std.typecons.scoped for that, and the deprecated scope 
classes. Also see deadalnix's `isolated` proposal:


http://forum.dlang.org/thread/yiwcgyfzfbkzcavuq...@forum.dlang.org

But I'm afraid your suggestion is unsafe: There also needs to be 
a way to guarantee that no references to the scoped object exist 
when it is destroyed.


I also don't think it's a good idea to have a hard-coded list of 
types that are scoped. This either needs to be specified by the 
user of the type (at instantion) and then become part of the type 
(like const), or it needs to be specified by the implementer of 
the type, as a requirement that it can only be instantiated as a 
scoped type, or as garbage-collected or reference-counted or ...

Re: D For A Web Developer


On 2014-05-02 19:25, brad clawsie wrote:


this has been the fundamental issue for me. its not just missing libs,
its libs that are surfaced via a C-binding, which in my limited


I've had problems with ImageMagick, basically every time. But that's the 
only one I can think of, at least for now.


--
/Jacob Carlborg

Re: GC vs Resource management.


On Sunday, 4 May 2014 at 11:42:14 UTC, Marc Schütz wrote:

On Saturday, 3 May 2014 at 12:28:03 UTC, monnoroch wrote:
Back to the dtors: i understand, that all the stuff you 
propose could make GC faster, simpler, and cooler,


Note that this is _not_ the motivation behind the discussed 
changes. It's primarily about correctness and consistency. What 
we currently have if something that doesn't work at all in some 
cases (dynamic arrays, new with structs), and where it does 
work (class destructors) users have to conform to various rules 
as to what is allowed in a GC-called destructor (e.g. don't 
access references to other objects, don't throw (?), don't use 
thread-local globals, any probably many more subtle rules), 
that mostly cannot be enforced statically or even at runtime, 
potentially causing silent corruption if you fail to comply 
with them. This is IMO more than enough motivation to do 
something about it.


Case in point:
http://forum.dlang.org/thread/vlnjgtdmyolgoiofn...@forum.dlang.org

Re: More radical ideas about gc and reference counting

2014-05-04 Thread Michel Fortin via Digitalmars-d


On 2014-05-04 09:00:45 +, Marc Schütz schue...@gmx.net said:


On Saturday, 3 May 2014 at 11:12:56 UTC, Michel Fortin wrote:
Or turn the rule on its head: make it so having a destructor makes the 
heap memory block reference counted. With this adding a destructor 
always cause deterministic destruction.


The compiler knows statically whether a struct has a destructor. For a 
class you need a runtime trick because the root object which can be 
either. Use a virtual call or a magic value in the reference count 
field to handle the reference count management. You also need a way to 
tag a class to be guarantied it has no derived class with a destructor 
(to provide a static proof for the compiler it can omit ARC code), 
perhaps @disable ~this().


Then remains the problem of cycles. It could be a hard error if the 
destructor is @safe (error thrown when the GC collects it). The 
destructor could be allowed to run (in any thread) if the destructor is 
@system or @trusted.


The interesting thing with this is that the current D semantics are 
preserved, destructors become deterministic (except in the presence of 
cycles, which the GC will detect for you), and if you're manipulating 
pointers to pure memory (memory blocks having no destructor) there's no 
ARC overhead. And finally, no new pointer attributes; Walter will like 
this last one.


This is certainly also an interesting idea, but I suspect it is bound 
to fail, simply because it involves ARC. Reference counting always 
makes things so much more complicated... See for example the cycles 
problem you mentioned: If you need a GC for that, you cannot guarantee 
that the objects will be collected, which was the reason to introduce 
ARC in the first place. Then there are the problems with shared vs. 
thread-local RC (including casting between the two), and arrays/slices 
of RC objects. And, of course, Walter doesn't like it ;-)


Arrays/slices of RC objects would have to be reference counted too. 
That was part of Andrei's proposal.


Cycles break deterministic destruction and break the guaranty the 
destructor will be called because we're relying on the GC to free them, 
true. But today we have no guaranty at all, even when there's no cycle. 
It is possible to avoid cycles, and not that hard to do in a safe way 
when you have weak (auto-nulling) pointers (which D should have 
regardless of the memory management scheme). It's even easier if you 
use the GC to detect them. I'm proposing that GC calls to a @safe 
destructor be a hard error so you can use that error to fix your 
cycles. (Note that cycles with no destructor are perfectly fine though.)


As for shared vs. thread-local RC, Andrei proposed a solution for that earlier.
http://forum.dlang.org/post/ljrr8l$2dt1$1...@digitalmars.com

I know Walter doesn't like to have pointers variables do magic stuff. 
This is why I'm confining ARC behaviour to objects, structs, and arrays 
having destructors. Andrei's initial proposal was to completely remove 
destructors for objects; all the pointers that would have survived that 
proposal are still RC-free in mine except for non-final classes when 
you haven't explicitly disabled the destructor. (An alternative to 
disabling the destructor would be to use a different root object for 
RC, I just find that less desirable.)


Frankly, if we want destructors to work safely when on the heap, I 
don't see another way.


--
Michel Fortin
michel.for...@michelf.ca
http://michelf.ca

Re: D For A Web Developer

On Sun, 2014-05-04 at 13:44 +0200, Jacob Carlborg via Digitalmars-d
wrote:
 On 2014-05-03 21:51, Atila Neves wrote:
 
  This is why I started to learn Cucumber.
 
 Cucumber is for acceptance tests. There are also functional and 
 integration tests.

We could get into bikeshedding here,…

Cucumber is really for supporting BDD. These include functional and
integration tests as well as some varieties of acceptance test. It
doesn't just have acceptance tests.

-- 
Russel.
=
Dr Russel Winder  t: +44 20 7585 2200   voip: sip:russel.win...@ekiga.net
41 Buckmaster Roadm: +44 7770 465 077   xmpp: rus...@winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

Re: python vs d

2014-05-04 Thread Suliman via Digitalmars-d

Here on Forum I had found 2 interesting link. I decided to put 
it's here. Maybe it would be helpful

http://codegolf.stackexchange.com/questions/26323/how-slow-is-python-really-or-how-fast-is-your-language
http://codegolf.stackexchange.com/questions/26371/how-slow-is-python-really-part-ii

Re: python vs d

2014-05-04 Thread Joseph Rushton Wakeling via Digitalmars-d


Suliman:
Here on Forum I had found 2 interesting link. I decided to put 
it's here. Maybe it would be helpful

http://codegolf.stackexchange.com/questions/26323/how-slow-is-python-really-or-how-fast-is-your-language
http://codegolf.stackexchange.com/questions/26371/how-slow-is-python-really-part-ii


See:
http://forum.dlang.org/thread/pvojsrqmaksqwokue...@forum.dlang.org

Bye,
bearophile

Re: The Current Status of DQt


On 03/05/14 13:00, w0rp via Digitalmars-d wrote:

Hello everyone. From time to time, people ask in the newsgroup and also IRC
about Qt bindings for D, so I thought it would be a good idea to give people an
update on where my own bindings stand. First, if you want to take a look at my
code as it stands, you can get it here.

https://github.com/w0rp/dqt


First of all, it's fantastic that you're working on this.  Many thanks!

I won't comment on the details of the code; it's outside my area of experience, 
but I do think that Qt support is something that will be very important for 
adoption of D.



So, I am eager to hear what people think about all of this. Does anyone like the
work that I have done, and will it be useful? Have I committed some terrible
crime against nature, for which I must be punished? Does anyone have any ideas
about things that could be improved, or where to go next? Please, let me know.


A remark about your README: it lists as dependencies A recent Qt 4 version, 
like Qt 4.8.


I think you should be far more explicit, far earlier in the README, about 
exactly which Qt versions are supported.  Pretty much the first question I had 
on my mind, clicking on the link to your project, was, Is this going to support 
Qt 5.x ... ?


I also think that a focus on 4.x rather than 5.x is likely to be 
counter-productive.  5.2+ seems to be where all the interest is these days.



My progress has been somewhat slow in writing all of this, as I have been
writing it mostly in the evenings, moslty during weekdays, after my day job.
I've also been running into problems for which there are no easy answers,
possibly no documentation for, and so on. So a great deal of my time has been
spent more on thinking of what to do next, rather than writing anything. (It
took me possibly months to come up with the resource management scheme that I
have now.)


I can sympathize with your predicament as I often find myself in the same 
position.  The good side is that the time spent thinking in this way is usually 
very productive when one comes to finally sit down and write :-)

Re: The Current Status of DQt


On Sunday, 4 May 2014 at 09:21:30 UTC, Marc Schütz wrote:

A meta question, not related to your specific implementation:

While Qt is certainly the most powerful and comprehensive 
portable GUI framework, it also contains lots of code that's 
not related to user interfaces: strings, multi-threading 
support, file abstractions, containers, databases interfaces, 
etc. In short: Things which properly should be part of the 
standard library, but aren't (widely) available, or not working 
well enough in C++.


Now, Qt depends on those, and if you want to interact with it, 
you need to use them in your own code. Doesn't that cause a lot 
of friction? Just as a random example, QListView can take its 
elements from a data source (model). But in D, that data 
source might be idiomatically implemented as a range, so it has 
to be adapted first. Or, a more frequent thing: char[] vs. 
QString.


Could it be a better strategy to only re-use low-level 
functionality of Qt internally, but provide more idiomatic 
public D interfaces for it?


I'm only interested in getting the GUI parts of Qt to work. A lot 
of the features of Qt exist I think because there wasn't a 
reasonable portable alternative in C++ at the time. I think with 
D, there are or will be better alternatives than what Qt offers 
for certain things. (Like a threading implementation, for 
instance.) So my interest in supporting Qt classes only extends 
as far as getting GUI building to work. So I want to use native D 
types like 'string' and 'int[]' as much as possible, rather than 
Qt's types.


I believe that some wrapping can be avoided. I haven't attempted 
it yet, but I believe that it should be possible to recreate 
value types from Qt in D, and pass them over such that the data 
matches up to what C++ expects. Then basic things like what to do 
with a bunch of QPoint values can be dealt with in idiomatic D 
ways.

Re: The Current Status of DQt


On Sunday, 4 May 2014 at 13:08:38 UTC, Joseph Rushton Wakeling via
A remark about your README: it lists as dependencies A recent 
Qt 4 version, like Qt 4.8.


I think you should be far more explicit, far earlier in the 
README, about exactly which Qt versions are supported.  Pretty 
much the first question I had on my mind, clicking on the link 
to your project, was, Is this going to support Qt 5.x ... ?


I also think that a focus on 4.x rather than 5.x is likely to 
be counter-productive.  5.2+ seems to be where all the interest 
is these days.


Qt 4.8 support comes largely from what is possible easily now. 
DQt uses SMOKE to wrap Qt. In particular smokeqt, which produces 
the SMOKE Qt bindings. You can get smokeqt for Qt 4 straight from 
most package managers (Arch Linux, Debian, Gentoo, ...) and it's 
also the current master version of the project on git. Someone 
has very recently been working on a Qt5 branch for smokeqt, so I 
expect Qt 5 support to come soon. Qt5 support for smokeqt will 
directly influcence Qt 5 support in DQt, and I might jump on it 
early for binding functions as slots. (A Qt 5 feature.)


Qt 4 support basically arises from what is easy to do right now. 
Supporting Qt 5 doesn't seem that far off. I went with Qt 4 for 
now because it's easier, and at this stage it's more important to 
work with something that can actually work and learn from that, 
than to try and work with something which might not actually work 
at all.

Re: Scenario: OpenSSL in D language, pros/cons

2014-05-04 Thread Meta via Digitalmars-d


On Sunday, 4 May 2014 at 08:34:20 UTC, Daniele M. wrote:

I have read this excellent article by David A. Wheeler:

http://www.dwheeler.com/essays/heartbleed.html

And since D language was not there, I mentioned it to him as a 
possible good candidate due to its static typing and related 
features.


However, now I am asking the community here: would a D 
implementation (with GC disabled) of OpenSSL have been free 
from Heartbleed-type vulnerabilities? Specifically 
http://cwe.mitre.org/data/definitions/126.html and 
http://cwe.mitre.org/data/definitions/20.html as David mentions.


I find this perspective very interesting, please advise :)


While D is a somewhat safer language by *default*, it makes it 
fairly easy to escape from the safe part of the language and 
write unsafe code (array bounds checking can be turned off even 
for @safe code). Seeing as the OpenSSL devs went as far as to 
write an a buggy, custom implementation of malloc for a speed 
gain, turning off array bounds checking and ignoring @safe seems 
like the first thing they would do. The only language I would 
really trust is one in which it is impossible to write unsafe 
code, because you can then know that the developers can't use 
such unsafe hacks, even if they wanted to.

Re: The Current Status of DQt

On Sun, 2014-05-04 at 13:15 +, w0rp via Digitalmars-d wrote:
[…]
 I'm only interested in getting the GUI parts of Qt to work. A lot 
 of the features of Qt exist I think because there wasn't a 
 reasonable portable alternative in C++ at the time. I think with 
 D, there are or will be better alternatives than what Qt offers 
 for certain things. (Like a threading implementation, for 
 instance.) So my interest in supporting Qt classes only extends 
 as far as getting GUI building to work. So I want to use native D 
 types like 'string' and 'int[]' as much as possible, rather than 
 Qt's types.

I guess a dark corner is going to be GUI network applications (which I
have as one of my pet projects). Qt supports GUI and network on the same
event loop, which is both good and terrible. For PyQt5 with a single
thread (and a GIL) it is great. For languages such as D and C++ which
can manage multiple kernel threads on multicore machines I would think
separate even loops would be better (but I don't yet have any data, I am
just beginning to try this idea out – it is likely that many people have
already trodden this route…) For this latter case the D architecture I
have in mind is vibe.d for network and GtkD or QtD^H^H^HDQt for GUI. I
suspect to do this sensibly needs signals for Qt.
[…]
-- 
Russel.
=
Dr Russel Winder  t: +44 20 7585 2200   voip: sip:russel.win...@ekiga.net
41 Buckmaster Roadm: +44 7770 465 077   xmpp: rus...@winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder

Re: The Current Status of DQt

2014-05-04 Thread Xavier Bigand via Digitalmars-d


Le 04/05/2014 11:21, Marc Schütz schue...@gmx.net a écrit :

A meta question, not related to your specific implementation:

While Qt is certainly the most powerful and comprehensive portable GUI
framework, it also contains lots of code that's not related to user
interfaces: strings, multi-threading support, file abstractions,
containers, databases interfaces, etc. In short: Things which properly
should be part of the standard library, but aren't (widely) available,
or not working well enough in C++.

Now, Qt depends on those, and if you want to interact with it, you need
to use them in your own code. Doesn't that cause a lot of friction? Just
as a random example, QListView can take its elements from a data source
(model). But in D, that data source might be idiomatically implemented
as a range, so it has to be adapted first. Or, a more frequent thing:
char[] vs. QString.

Could it be a better strategy to only re-use low-level functionality of
Qt internally, but provide more idiomatic public D interfaces for it?


With a friend we created the DQuick project cause of our major interest 
of the QtQuick (also called QML) part of Qt framework and also for the 
reason you invoke. For us phobos already aim to implement same things 
than QtCore, and wrapping primitive types of Qt seems hard and will 
introduce design constraint and poor performances. It's certainly bad 
for a longterm vision.


IMO DQt will be interesting for those looking for a stable GUI library 
based on widget essentially.
DQuick have no chance to be usable in production before years if we 
continue to develop it.
We are back after a long break, it's really hard to stay motivated cause 
we have some difficulties with points aren't fixed in D, mainly GC 
issues, signals,... My friend just find get the property binding working 
in D code, but he doesn't like his code :-).


I see DQt/gtkD and DQuick as complementary projects. I hope we will 
capable to provide good enough GUI libraries to D community shortly.

Re: Scenario: OpenSSL in D language, pros/cons

2014-05-04 Thread Nick Sabalausky via Digitalmars-d


On 5/4/2014 9:29 AM, Meta wrote:


While D is a somewhat safer language by *default*, it makes it fairly
easy to escape from the safe part of the language and write unsafe code


Yea, I'm finding that in some ways, D accidentally encourages 
@system/@trusted code. For example, if you need some sensitive data 
zeroed out when done, and for whatever reason you aren't able to just 
have it live on the stack (to use RAII), then you need RefCounted which 
AFAICS throws any chance of being @safe out the window.


Maybe RefCounted could somehow take advantage of Unique to provide @safe-ty?

Re: D For A Web Developer

2014-05-04 Thread Nick Sabalausky via Digitalmars-d


On 5/4/2014 12:34 AM, Jonathan M Davis via Digitalmars-d wrote:


Regardless, unittest blocks don't really put any restrictions on what kind of
code can go in them, and I'd prefer that that stay the case. The discussion
on parallelizing unit tests threatens that on some level, but as long as we
have the means to mark unittest blocks in some manner that tells the test
runner not to run them in parallel with any other unittest blocks, then I
think that we should be fine on that front.



Yes, this.

Re: D For A Web Developer

2014-05-04 Thread Meta via Digitalmars-d


On Sunday, 4 May 2014 at 10:04:12 UTC, bearophile wrote:

Danny Weldon:

If somebody has some time, they could post a solution in D to 
this problem:


http://codegolf.stackexchange.com/questions/26323/how-slow-is-python-really-or-how-fast-is-your-language

It all helps to get the language visible.


Here are my solutions:
http://forum.dlang.org/thread/pvojsrqmaksqwokue...@forum.dlang.org

I don't have a Stackexchange account. You are welcome to post 
the two solutions (of two different problems).


Bye,
bearophile


I'm not able to run either version on DPaste. It fails to compile 
on both examples with no error message. Both example compile fine 
locally with DMD 2.065, however.


Your C++ translation: ~277ms
Your second version:  ~2.34ms/round

I just compiled and ran each binary a few times, so these might 
not be good measurements. I compiled both versions without 
-inline, as they ran slower with it enabled. LDC might do a 
better job with this.

Re: The Current Status of DQt


On Sunday, 4 May 2014 at 13:32:13 UTC, Xavier Bigand wrote:
With a friend we created the DQuick project cause of our major 
interest of the QtQuick (also called QML) part of Qt framework 
and also for the reason you invoke. For us phobos already aim 
to implement same things than QtCore, and wrapping primitive 
types of Qt seems hard and will introduce design constraint and 
poor performances. It's certainly bad for a longterm vision.


IMO DQt will be interesting for those looking for a stable GUI 
library based on widget essentially.
DQuick have no chance to be usable in production before years 
if we continue to develop it.
We are back after a long break, it's really hard to stay 
motivated cause we have some difficulties with points aren't 
fixed in D, mainly GC issues, signals,... My friend just find 
get the property binding working in D code, but he doesn't like 
his code :-).


I see DQt/gtkD and DQuick as complementary projects. I hope we 
will capable to provide good enough GUI libraries to D 
community shortly.


Best of luck to you guys. I encourage as many people as possible 
to give writing D GUI libraries a go, and perhaps we can all 
learn from each other.

Re: D For A Web Developer


Meta:


Your C++ translation: ~277ms
Your second version:  ~2.34ms/round


Both D programs are translations of C++ programs.



LDC might do a better job with this.


I have developed those two programs using ldc2, so the usage of 
ldc2 is encouraged, and inlining is necessary for both programs. 
Generally don't use DMD for benchmark contests like this one.


Bye,
bearophile

Re: D For A Web Developer

2014-05-04 Thread Joseph Rushton Wakeling via Digitalmars-d

The Nimrod version partially unrolls the recursion 4 times. How 
hard is this to do in D?


http://codegolf.stackexchange.com/questions/26459/how-high-can-you-go-a-codingalgorithms-challenge

Bye,
bearophile

Re: isUniformRNG

2014-05-04 Thread Nick Sabalausky via Digitalmars-d


On 5/4/2014 3:47 AM, Joseph Rushton Wakeling via Digitalmars-d wrote:


About a more general isRNG template: can you be more precise about
what you are interested in achieving with this?  Generally speaking I
would find it rather dangerous to go passing around sources of
randomness without having some understanding of their properties :-)



Fair enough point, I'll explain my situation:

On a general level, I'm trying to grok the whole intent of isUniformRNG 
and see whether or not anything else may ever be needed in addition to 
isUniformRNG. I'm not trying to push an isRNG, just trying to 
understand std.random's current intentions and reasoning, so I know how 
to work with it appropriately.


But more immediately, since Phobos lacks a crypto-secure RNG, I'm 
implementing NIST's Hash_DRBG (backed by the OS-provided 
RtlGenRandom/CryptGenRandom and /dev/random as entropy sources). 
Hopefully I can get it into a Phobos-acceptable state.


Now, I can follow the spec for the Hash_DRBG algorithm well enough, but 
I'm not really solid on random-number theory, so I can't be certain 
whether or not isUniformRNG is appropriate for this. I would assume 
yes, but I don't want to assume. Hence my inquiries.

Re: isUniformRNG


On 04/05/14 16:28, Nick Sabalausky via Digitalmars-d wrote:

On a general level, I'm trying to grok the whole intent of isUniformRNG and see
whether or not anything else may ever be needed in addition to isUniformRNG. I'm
not trying to push an isRNG, just trying to understand std.random's current
intentions and reasoning, so I know how to work with it appropriately.


Yea, it's a good question.  I think I would answer it as follows.

From a design point of view, it's helpful to distinguish between code that acts 
as a fundamental source of (usually pseudo-) randomness, versus stuff that 
transforms the output of those random sources into other forms (say, 
uniformly-distributed floating point numbers, or rolls of a 6-sided die, or 
permutations of an array, or whatever else).


In other words, it's helpful to distinguish between _generators_ of randomness, 
versus _users_ of randomness.


Now, in principle, there are many different kinds of random generator available, 
not all of them drawing from a uniform distribution.  But again, from a design 
point of view, it's helpful to constrain the range of functionality assumed, and 
uniformly-distributed random numbers are a very good foundation from which to 
build, because they can reliably be used to generate numbers drawn from any 
other probability distribution.


On top of that, it's useful to have the constraint of a distribution on a closed 
interval that has explicitly included minimum and maximum values, because that 
greatly simplifies the checks and assumptions one has to make in writing 
functions that make use of the random generator.


Hence you come to this definition of a uniform random number generator, which 
is the fundamental building block of all other random-number functionality.
What the isUniformRNG template is _really_ asking is, Is this going to give me 
a valid source of randomness to plug in to all the other 
random-number-generation functions, so that they'll do what I expect?


Simple example of how that can be important: suppose you have a random generator 
that gives you uniform numbers in the half-open interval [min, max).  Now run 
that through std.random.uniform![]().  You will not in fact get random numbers 
drawn from the closed interval you have specified.



But more immediately, since Phobos lacks a crypto-secure RNG, I'm implementing
NIST's Hash_DRBG (backed by the OS-provided RtlGenRandom/CryptGenRandom and
/dev/random as entropy sources). Hopefully I can get it into a
Phobos-acceptable state.



That's great -- I really look forward to seeing your work :-)  Can you give me 
some links to the spec?



Now, I can follow the spec for the Hash_DRBG algorithm well enough, but I'm not
really solid on random-number theory, so I can't be certain whether or not
isUniformRNG is appropriate for this. I would assume yes, but I don't want to
assume. Hence my inquiries.


I don't know that this necessarily needs too much in the way of random-number 
theory, but there are 3 questions that really need answering here:


   * What's the type of number that this algorithm produces?

   * Are these numbers uniformly distributed?

   * What's the min and max values produced, and are these
 inclusive bounds, i.e. are the random numbers drawn
 from the closed [min, max] interval?

Can you clarify?

Re: Running Phobos unit tests in threads: I have data

2014-05-04 Thread Andrei Alexandrescu via Digitalmars-d


On 5/4/14, 3:06 AM, Russel Winder via Digitalmars-d wrote:

On Sun, 2014-05-04 at 08:47 +, Atila Neves via Digitalmars-d wrote:

Like I mentioned afterwards, I tried a different number of
threads. On my machine, at least, std.parallelism.totalCPUs
returns 8, the number of virtual cores. As it should.


If you can create a small example of the problem, and I can remember how
to run std.parallelism as a separate module, I can try and take a look
at this later next week.


This is an awesome offer, Russel. Thanks! -- Andrei

Re: Running Phobos unit tests in threads: I have data

2014-05-04 Thread Andrei Alexandrescu via Digitalmars-d


On 5/4/14, 1:44 AM, Atila Neves wrote:

On Saturday, 3 May 2014 at 22:46:03 UTC, Andrei Alexandrescu wrote:

On 5/3/14, 2:42 PM, Atila Neves wrote:

gdc gave _very_ different results. I had to use different modules
because at some point tests started failing, but with gdc the threaded
version runs ~3x faster.

On my own unit-threaded benchmarks, running the UTs for Cerealed over
and over again was only slightly slower with threads than without. With
dmd the threaded version was nearly 3x slower.


Sounds like a severe bug in dmd or dependents. -- Andrei


Seems like it. Just to be sure I swapped ld.gold for ld.bfd and the
problem was still there.

I'm not entirely sure how to file this bug: with just my simple example
above?


The simpler the better. -- Andrei

Re: GC vs Resource management.

2014-05-04 Thread Andrei Alexandrescu via Digitalmars-d


On 5/4/14, 4:42 AM, Marc Schütz schue...@gmx.net wrote:

But I'm afraid your suggestion is unsafe: There also needs to be a way
to guarantee that no references to the scoped object exist when it is
destroyed.


Actually, it should be fine to call the destructor, then blast T.init 
over the object, while keeping the actual memory in the GC. This 
possible approach has come up a number of times, and I think it has 
promise. -- Andrei

Re: Running Phobos unit tests in threads: I have data

2014-05-04 Thread Joseph Rushton Wakeling via Digitalmars-d


On 04/05/14 09:49, Russel Winder via Digitalmars-d wrote:

(*) Physical cores are not necessarily the number reported by the OS due
to core hyperthreads. Quad core no hyperthreads, and dual core, two
hyperthreads per core, both get reported as four processor systems.
However if you benchmark them you get very, very different performance
characteristics.


Yup.  That bit me with a new laptop the first time I tried parallel programming 
with D :-)

Re: The Current Status of DQt

2014-05-04 Thread Gary Willoughby via Digitalmars-d


On Sunday, 4 May 2014 at 14:09:38 UTC, w0rp wrote:
Best of luck to you guys. I encourage as many people as 
possible to give writing D GUI libraries a go, and perhaps we 
can all learn from each other.


Done ;)

http://forum.dlang.org/thread/wdddgiowaidcojbrk...@forum.dlang.org

Re: Running Phobos unit tests in threads: I have data

2014-05-04 Thread safety0ff via Digitalmars-d

On Saturday, 3 May 2014 at 22:46:03 UTC, Andrei Alexandrescu 
wrote:

On 5/3/14, 2:42 PM, Atila Neves wrote:
gdc gave _very_ different results. I had to use different 
modules
because at some point tests started failing, but with gdc the 
threaded

version runs ~3x faster.

On my own unit-threaded benchmarks, running the UTs for 
Cerealed over
and over again was only slightly slower with threads than 
without. With

dmd the threaded version was nearly 3x slower.


Sounds like a severe bug in dmd or dependents. -- Andrei


This reminds me of when I was parallelizing a project euler 
solution: atomic access was so much slower on DMD that it made 
performance worse than the single threaded version for one stage 
of the program.


I know that std.parallelism does make use of core.atomic under 
the hood, so this may be a factor when using DMD.

Re: isUniformRNG

2014-05-04 Thread Nick Sabalausky via Digitalmars-d

On 5/4/2014 11:38 AM, Joseph Rushton Wakeling via Digitalmars-d wrote:

On 04/05/14 16:28, Nick Sabalausky via Digitalmars-d wrote:

On a general level, I'm trying to grok the whole intent of
isUniformRNG and see
whether or not anything else may ever be needed in addition to
isUniformRNG. I'm
not trying to push an isRNG, just trying to understand std.random's
current
intentions and reasoning, so I know how to work with it appropriately.

Yea, it's a good question. I think I would answer it as follows.

[...]

Ok, I see. Thanks.

But more immediately, since Phobos lacks a crypto-secure RNG, I'm
implementing
NIST's Hash_DRBG (backed by the OS-provided
RtlGenRandom/CryptGenRandom and
/dev/random as entropy sources). Hopefully I can get it into a
Phobos-acceptable state.

That's great -- I really look forward to seeing your work :-) Can you
give me some links to the spec?

The spec for Hash_DRBG is in NIST Special Publication 800-90A which
also includes a few other crypto RNGs (including, unfortunately, the one
with a known NSA backdoor, Dual_EC_DRBG, but it's easy enough to just
not bother implementing that one):

http://csrc.nist.gov/publications/nistpubs/800-90A/SP800-90A.pdf

That paper is linked to from here (seems to be down right now though):

http://csrc.nist.gov/groups/ST/toolkit/random_number.html

Which I found from here:

http://en.wikipedia.org/wiki/Cryptographically_secure_pseudorandom_number_generator#Standards

Now, I can follow the spec for the Hash_DRBG algorithm well enough,
but I'm not
really solid on random-number theory, so I can't be certain whether or
not
isUniformRNG is appropriate for this. I would assume yes, but I
don't want to
assume. Hence my inquiries.

I don't know that this necessarily needs too much in the way of
random-number theory, but there are 3 questions that really need
answering here:

* What's the type of number that this algorithm produces?

Just a string of random bits. Effectively unsigned integers.

* What's the min and max values produced, and are these
inclusive bounds, i.e. are the random numbers drawn
from the closed [min, max] interval?

It's all based on SHA hashes (of seeds, internal state, and some other
stuff) and it doesn't explicitly exclude any output values. So if SHA is
working as expected, then yea, I'd imagine it should be a closed
interval over [0, 2^numBitsRequested - 1] where numBitsRequested is
permitted by the spec to be any positive integer.

* Are these numbers uniformly distributed?

This is the part I've been a little unclear on, although maybe I'm just
being paranoid about it. Doesn't a uniform distribution carry certain
reliable statistical properties? I don't know whether that could be used
to improve the likelihood of guessing future values. If so, then maybe
the algorithm is designed to avoid that?

Then again, wouldn't the only alternative to uniform distribution be a
weighted distribution? I can't imagine an RNG intended for crypto would
be deliberately weighted (unless maybe there were some randomness to the
weights...if that even makes any sense at all).

Maybe I'm just overthinking it?

Re: The Current Status of DQt

2014-05-04 Thread Joseph Rushton Wakeling via Digitalmars-d


On Sunday, 4 May 2014 at 16:19:32 UTC, Gary Willoughby wrote:

On Sunday, 4 May 2014 at 14:09:38 UTC, w0rp wrote:
Best of luck to you guys. I encourage as many people as 
possible to give writing D GUI libraries a go, and perhaps we 
can all learn from each other.


Done ;)

http://forum.dlang.org/thread/wdddgiowaidcojbrk...@forum.dlang.org


I just read your announcement and this is very good.

Re: The Current Status of DQt

2014-05-04 Thread Andrei Alexandrescu via Digitalmars-d


On 5/4/14, 9:19 AM, Gary Willoughby wrote:

On Sunday, 4 May 2014 at 14:09:38 UTC, w0rp wrote:

Best of luck to you guys. I encourage as many people as possible to
give writing D GUI libraries a go, and perhaps we can all learn from
each other.


Done ;)

http://forum.dlang.org/thread/wdddgiowaidcojbrk...@forum.dlang.org


Worth a reddit announcement tomorrow morning? -- Andrei

Re: isUniformRNG


On 04/05/14 19:42, Nick Sabalausky via Digitalmars-d wrote:

Just a string of random bits. Effectively unsigned integers.


Ahh, OK.  So in practice you can probably template it on an unsigned integral 
type (which could include bool) and it'll just take the appropriate number of 
bits from the stream, no ... ?  Cf. what I did with /dev/urandom etc.:

https://github.com/WebDrake/std.random2/blob/master/std/random2/device.d#L122


It's all based on SHA hashes (of seeds, internal state, and some other stuff)
and it doesn't explicitly exclude any output values. So if SHA is working as
expected, then yea, I'd imagine it should be a closed interval over [0,
2^numBitsRequested - 1] where numBitsRequested is permitted by the spec to be
any positive integer.


Yea, sounds right to me.


This is the part I've been a little unclear on, although maybe I'm just being
paranoid about it. Doesn't a uniform distribution carry certain reliable
statistical properties? I don't know whether that could be used to improve the
likelihood of guessing future values. If so, then maybe the algorithm is
designed to avoid that?

Then again, wouldn't the only alternative to uniform distribution be a weighted
distribution? I can't imagine an RNG intended for crypto would be deliberately
weighted (unless maybe there were some randomness to the weights...if that even
makes any sense at all).

Maybe I'm just overthinking it?


Probably :-)  Let's put it this way: if you think in terms of the individual 
bits being generated, there obviously has to be, from the point of view of the 
user of the algorithm, no way to decide which bit value is more likely, which 
corresponds to a uniform distribution of individual bit values.  And that in 
turn will map to a uniform distribution of bit sequences of any length.

Re: isUniformRNG

2014-05-04 Thread Joseph Rushton Wakeling via Digitalmars-d


On 04/05/14 20:10, Joseph Rushton Wakeling via Digitalmars-d wrote:

Probably :-)  Let's put it this way:


... we can always empirically verify the uniformity of the distribution :-)

Re: Progress on Adam Wilson's Graphics API?

2014-05-04 Thread Jonas Drewsen via Digitalmars-d


Just had a quick look at the source code.

If this is to be something like the official gfx library wouldn't 
it make sense to follow the phobos coding style?

For example struct Size instead of struct SIZE

/Jonas

Re: [OT] Slides from 2014 International Symposium on Code Generation and Optimization

2014-05-04 Thread Joseph Rushton Wakeling via Digitalmars-d


Paulo Pinto:


http://cgo.org/cgo2014/conference/program/


I like the Partial escape analysis for Java (the compiler doesn't 
heap allocate an object only in the most frequent code paths 
inside a function), and optimization usable for D too:

http://cgo.org/cgo2014/wp-content/uploads/2013/05/Partial_Escape_Analysis.pdf

Bye,
bearophile

Re: isUniformRNG

2014-05-04 Thread Nick Sabalausky via Digitalmars-d


On 5/4/2014 2:10 PM, Joseph Rushton Wakeling via Digitalmars-d wrote:

On 04/05/14 19:42, Nick Sabalausky via Digitalmars-d wrote:

Just a string of random bits. Effectively unsigned integers.


Ahh, OK.  So in practice you can probably template it on an unsigned
integral type (which could include bool) and it'll just take the
appropriate number of bits from the stream, no ... ?  Cf. what I did
with /dev/urandom etc.:
https://github.com/WebDrake/std.random2/blob/master/std/random2/device.d#L122



Well, Hash_DRBG isn't really a normal stream since, based on my reading 
of its spec, it sounds like (for example) requesting one byte four times 
will give a different result than requesting four bytes all at once 
(assuming you're starting from the same internal state and there's no 
reseeding).


But aside from that minor detail, yes, that's essentially correct. And 
much like /dev/(u)random, you could even make the number of bytes/bits 
requested a per-call runtime parameter (although that would diverge from 
the existing std.random interfaces and would require either allocating 
or taking an output sink, so I don't know whether I'll bother).




Then again, wouldn't the only alternative to uniform distribution be a
weighted
distribution? I can't imagine an RNG intended for crypto would be
deliberately
weighted (unless maybe there were some randomness to the weights...if
that even
makes any sense at all).

Maybe I'm just overthinking it?


Probably :-)  Let's put it this way: if you think in terms of the
individual bits being generated, there obviously has to be, from the
point of view of the user of the algorithm, no way to decide which bit
value is more likely, which corresponds to a uniform distribution of
individual bit values.  And that in turn will map to a uniform
distribution of bit sequences of any length.


Yea. Plus, this doc about testing these crypto PRNGs...

http://csrc.nist.gov/groups/ST/toolkit/rng/documents/SP800-22rev1a.pdf

...does mention the importance of uniformity.

So I think it's probably safe to figure this is a uniform distribution 
unless some expert chimes in and says otherwise.


Thanks for the help.

Re: isUniformRNG


On 04/05/14 20:56, Nick Sabalausky via Digitalmars-d wrote:

So I think it's probably safe to figure this is a uniform distribution unless
some expert chimes in and says otherwise.

Thanks for the help.


You're very welcome.  Keep me posted on how things go with your implementation! 
:-)

Re: isUniformRNG

2014-05-04 Thread Nick Sabalausky via Digitalmars-d


On 5/4/2014 3:26 PM, Joseph Rushton Wakeling via Digitalmars-d wrote:

On 04/05/14 20:56, Nick Sabalausky via Digitalmars-d wrote:

So I think it's probably safe to figure this is a uniform distribution
unless
some expert chimes in and says otherwise.

Thanks for the help.


You're very welcome.  Keep me posted on how things go with your
implementation! :-)



Will do. Speaking of, this algo uses SHA-1 and SHA-2 (depending on the 
desired crypto strength of the psuedo-randomness). SHA-1 of course has 
been in Phobos for awhile. I've recently expanded it to support SHA-2, 
pending sufficient Phobos pull code reviews:


https://github.com/D-Programming-Language/phobos/pull/2129

Re: isUniformRNG

2014-05-04 Thread Joseph Rushton Wakeling via Digitalmars-d


On 04/05/14 21:26, Joseph Rushton Wakeling via Digitalmars-d wrote:

On 04/05/14 20:56, Nick Sabalausky via Digitalmars-d wrote:

So I think it's probably safe to figure this is a uniform distribution unless
some expert chimes in and says otherwise.

Thanks for the help.


You're very welcome.  Keep me posted on how things go with your implementation! 
:-)


I should also be thanking you -- this reminded me that I needed to make some key 
changes to my std.random2 proposal, including this stricter isUniformRNG template:

https://github.com/WebDrake/std.random2/commit/a071d8d182eb28516198bb292a0b45f86f4425fe

How does this look to people?  And would there be interest in seeing this land 
in the existing std.random ... ?

Enforced @nogc for dtors?

If we keep class destructors in D, is it a good idea to require 
them to be @nogc?


This post comes after this thread in D.learn:
http://forum.dlang.org/thread/vlnjgtdmyolgoiofn...@forum.dlang.org

Bye,
bearophile

Re: Scenario: OpenSSL in D language, pros/cons

2014-05-04 Thread Daniele M. via Digitalmars-d

On Sunday, 4 May 2014 at 10:23:38 UTC, Jonathan M Davis via 
Digitalmars-d wrote:

On Sun, 04 May 2014 08:34:19 +
Daniele M. via Digitalmars-d digitalmars-d@puremagic.com 
wrote:



I have read this excellent article by David A. Wheeler:

http://www.dwheeler.com/essays/heartbleed.html

And since D language was not there, I mentioned it to him as a
possible good candidate due to its static typing and related
features.

However, now I am asking the community here: would a D
implementation (with GC disabled) of OpenSSL have been free 
from

Heartbleed-type vulnerabilities? Specifically
http://cwe.mitre.org/data/definitions/126.html and
http://cwe.mitre.org/data/definitions/20.html as David 
mentions.


I find this perspective very interesting, please advise :)


@safe code protects against indexing an array out-of-bounds. 
So, if OpenSSL
had been implemented in D, and its heartbeat feature used @safe 
code, then
heartbleed would not have been possible. As soon as an attempt 
was made to
index passed the end of the array, it would have thrown a 
RangeError and

killed the program.


This is what I thought too.

Now, even if OpenSSL had been implemented in D, if it had used 
@system or
@trusted code for its heartbeat feature, then it could have had 
the bug just
as easily in D as it did in C. And given all of the horrible 
practices in
OpenSSL, I very much doubt that having it written in D would 
have prevented
much, because anyone making the choices that the OpenSSL folks 
have been
making would likely have ended up with horrible D code which 
was mostly
@system and probably doing all kinds of things that are 
inherently risky,

forgoeing many of the benefits that D provides.

I think that it's safe to say that D makes it easier to catch 
problems like
this, but it doesn't entirely prevent them, and bad programming 
practices
can pretty much always get around protections that the language 
provides
unless the language provides no ways around those protections 
(which isn't
the case in D, because it's a systems language and needs to 
provide low-level
access and features for those programs that need them - it just 
doesn't use

those by default).

I should have said: a D implementation of OpenSSL *without* the 
malloc sorcerery, sorry for leaving it out of OP. Reason I'd like 
to ignore that for this discussion is that I consider it a bug. 
And we are all in the post-heartbleed scenario now so it looks 
huge, but Theo De Raadt already said it all, nothing to add on it.


And then comes my next question: except for that malloc-hack, 
would it have been possible to write it in @safe D? I guess that 
if not, module(s) could have been made un-@safe. Not saying that 
a similar separation of concerns was not possible in OpenSSL 
itself, but that D could have made it less development-expensive 
in my opinion.


If I had more time, I'd actually be tempted to write an SSL 
implementation in
D, but even if I were to do an excellent job of it, it would 
still need to be
vetted by security experts to make sure that it didn't have 
horrible security
bugs in it (much as it would be likely that there would be 
fewer thanks to
the fact that it would be writen in D), and I suspect that it's 
the kind of
thing that many people aren't likely to trust because of how 
critical it is.

Nobody would expect/trust a single person to do this job :P
Working in an open source project would be best.

So, I don't know how good an idea it is at this point for 
someone to write
an implementation of SSL or TLS in D. Certainly, it's the type 
of thing where
we've generally tried to wrap existing C libraries in order to 
avoid having
to spend the time, effort, and expertise on in order to fully 
implement it
ourselves. The Go guys did it, but if I understand correctly, 
the fellow that
did it was one of the OpenSSL developers, so presumably he's 
already very
familiar with all of the ins and outs of SSL, and I don't know 
if any of us
here are (I'm certainly not - if I were doing it, I'd pretty 
much just have

to go off of its associated RFCs, for better or worse).


I wasn't aware of this effort, interesting experiment.

At this point though, if I were looking at using an existing 
implementation
of SSL, I'd probably be looking at GnuTLS rather than OpenSSL 
given how
horrible OpenSSL's codebase is. I don't know that GnuTLS is any 
better, but
it wouldn't be hard for it to be. OpenSSL is horrible both with 
regards to
its implementation and its API, and we'd all be better off if 
something

better replaced it (be it GnuTLS or something else).

I would pick LibreSSL:
http://www.libressl.org/

And merge patches as their progress goes on.


Unfortunately, even if
something better _were_ written in D, it's probably only the D 
folks who
would benefit, since it's not terribly likely at this point 
that very many
folks are going to wrap a D library in order to use it another 
language.


Here I don't completely agree: if we can have a

Re: Scenario: OpenSSL in D language, pros/cons

2014-05-04 Thread Daniele M. via Digitalmars-d


On Sunday, 4 May 2014 at 13:29:34 UTC, Meta wrote:

On Sunday, 4 May 2014 at 08:34:20 UTC, Daniele M. wrote:

I have read this excellent article by David A. Wheeler:

http://www.dwheeler.com/essays/heartbleed.html

And since D language was not there, I mentioned it to him as a 
possible good candidate due to its static typing and related 
features.


However, now I am asking the community here: would a D 
implementation (with GC disabled) of OpenSSL have been free 
from Heartbleed-type vulnerabilities? Specifically 
http://cwe.mitre.org/data/definitions/126.html and 
http://cwe.mitre.org/data/definitions/20.html as David 
mentions.


I find this perspective very interesting, please advise :)


While D is a somewhat safer language by *default*, it makes it 
fairly easy to escape from the safe part of the language and 
write unsafe code (array bounds checking can be turned off even 
for @safe code). Seeing as the OpenSSL devs went as far as to 
write an a buggy, custom implementation of malloc for a speed 
gain, turning off array bounds checking and ignoring @safe 
seems like the first thing they would do. The only language I 
would really trust is one in which it is impossible to write 
unsafe code, because you can then know that the developers 
can't use such unsafe hacks, even if they wanted to.


You are right, devs would eventually abuse everything possible, 
although it would make it for sure more visible: you cannot 
advertize an un-@safe library as @safe, although I agree that a 
lot depends from devs/users culture.

Re: The Current Status of DQt

2014-05-04 Thread Xavier Bigand via Digitalmars-d


Le 04/05/2014 18:19, Gary Willoughby a écrit :

On Sunday, 4 May 2014 at 14:09:38 UTC, w0rp wrote:

Best of luck to you guys. I encourage as many people as possible to
give writing D GUI libraries a go, and perhaps we can all learn from
each other.


Done ;)

http://forum.dlang.org/thread/wdddgiowaidcojbrk...@forum.dlang.org


It's a great news.

Re: More radical ideas about gc and reference counting

2014-05-04 Thread H. S. Teoh via Digitalmars-d

On Sat, May 03, 2014 at 10:48:47PM -0500, Caligo via Digitalmars-d wrote:
[...]
 Last but not least, currently there are two main ways for new features
 to make it into D/Phobos: you either have to belong to the inner
 circle, or have to represent some corporation that's doing something
 with D.

I'm sorry, but this is patently false. I am neither in the inner circle,
nor do I represent any corporation, yet I've had many changes pulled
into Phobos (including brand new code).

I can't say I'm perfectly happy with the D development process either,
but this kind of accusation is bordering on slander, and isn't helping
anything.


T

-- 
Why are you blatanly misspelling blatant? -- Branden Robinson

Re: More radical ideas about gc and reference counting

2014-05-04 Thread Caligo via Digitalmars-d

On Sun, May 4, 2014 at 12:22 AM, Andrei Alexandrescu via Digitalmars-d 
digitalmars-d@puremagic.com wrote:


 Mostly good points, but the bountysource program is an experiment by
 Facebook, not by myself. And (without me trying to speak on Facebook's
 behalf) it would be difficult to argue that Facebook doesn't understand
 FOSS or is out there to insult contributors. We're just experimenting with
 various angles.


If the bounty system was such a great idea, then every FOSS project would
be using it.  Now, hiring full-time engineers to work on a FOSS project,
that's an entirely different issue.  Besides, if someone is trying to
figure out how FOSS teams manage to become successful in regards to
development and all the associated technical and social complexities, then
all they have to do is study one of the million different FOSS projects out
there.  Many well known FOSS contributors have actually documented their
experience and knowledge of managing FOSS projects.



 The on/off switch may be a nice idea in the abstract but is hardly the
 perfect recipe to good language feature development; otherwise everybody
 would be using it, and there's not overwhelming evidence to that. (I do
 know it's been done a few times, such as the (in)famous new scoping rule
 of the for statement for C++ which has been introduced as an option by
 VC++.)


No, it's nothing abstract, and it's very practical and useful.  Rust has
such a thing, #![feature(X,Y,Z)].  So does Haskell, with {-# feature #-}.
 Even Python has __future__, and many others.


 I wonder how you've gotten the perception that one needs to be a member of
 the inner circle mafia to get things into D. Today's major contributors to
 D came from all over, without any preexisting relationship to anyone else,
 and their admission ticket has been getting work done. Could you please get
 into detail on how you view things? (I tried to look over your past posts
 to see a pattern of rejected contributions, but didn't find such.)



I wasn't trying to imply that contributions are rejected if contributors
are not members of a certain group.  I was just trying to say that it's
more likely for a _new_ feature to make it into D/Phobos if they are
proposed by members of the inner circle or someone representing a
corporation, probably because they become more noticeable, or because they
get lost in all the forum noise, I don't know.  I could be wrong, but
that's just how I perceive it.




 Actually I'd love to get you started so I'd understand your angle better.
 I'm sure we can do a lot better. One good thing Phobos reviews have done
 since we initiated them has been to prevent bad artifacts to make it into
 the library. We'd love to make it better. From what I saw witnessing
 similar processes (C++, Boost, Python, Scala) - they all have some sense of
 awkward self-importance to them upon the first look. I think that's the way
 such things work.



Here is an idea:  include new features in DMD/Phobos as soon as they
arrive, and make them part of the official binary release so that the
average D user can try them out.  Make sure they are marked as unstable,
and put a on/off switch on them (something like what Rust/Haskell have; not
a compiler switch).  If the feature receives no implementation bug reports
for X consecutive days AND no design bug reports for Y consecutive days,
then the feature is marked stable and officially becomes part of
DMD/Phobos.  The X and the Y can be decreased as D's number of users
increases over the years.  The whole idea is very much like farming: you
are planting seeds.  As the plants grow, some of them will not survive,
others will be destroyed, and some of them will take years to grow.  In any
case, you harvest the fruits when they are ready.

 Here are good starting values for X and Y:
X = 90 days
Y = 180 days

Re: Parallel execution of unittests

2014-05-04 Thread Walter Bright via Digitalmars-d


On 5/2/2014 4:02 AM, bearophile wrote:

Walter Bright:


You've already got it working with version, that's what version is for. Why
add yet another way to do it?


Because I'd like something better. It's an idiom that I have used many times
(around 15-20 times). I'd like the compiler (or build tool) to avoid me to
specify two times what the main module is.

Also, the current way to do it, in those modules I have to specify the module
name two times (once at the top and once at the bottom, unless I use some
compile-time syntheses of the version identifier from the current module name).


D has so many language features, we need a higher bar for adding new ones, 
especially ones that can be done straightforwardly with existing features.

The announcements are lacking something...

2014-05-04 Thread Xinok via Digitalmars-d

When I look through the announcements newsgroup, I see that 
there's a lot happening in the world of D. Development for the 
language has picked up a lot of pace in recent years. However, 
looking through the subject matter, I notice a common theme:


Libraries for D,
Compilers for D,
IDEs / Plugins for D,
Literature for D,
News about D...

That's all fine, but there's one thing that is desperately 
needed, which is announcements that aren't about things for D. 
We need more news about projects that are written in D but not 
for D, e.g. flint, Facebook's C++ linter written in D.


I know there are people out there using D for production, but 
it's a bit depressing that we rarely see this type of news in the 
D newsgroups. I'm sure we'll hear about some of these projects 
during the upcoming DConf, and I encourage those that do to post 
announcements and write articles about their projects around this 
time. And in general, if you, or anybody you know, is writing a 
project in D, please share the news with the world!

Re: Discusssion on the Discussion of the Design for a new GC

Am Wed, 30 Apr 2014 19:10:04 +
schrieb Orvid King blah38...@gmail.com:

 Just thought of a minor addition to the guidelines.
 
 While discussion of the naming of the public API should occur on 
 the newsgroup, discussion of the names of locals, or non-public 
 APIs should occur on Github.
 
 Any objections/concerns/improvements to this?

No and I think it comes natural that we don't need to discuss
internal variable names here. :)

-- 
Marco

Re: A few considerations on garbage collection

Am Wed, 30 Apr 2014 08:33:25 -0700
schrieb Andrei Alexandrescu seewebsiteforem...@erdani.org:

 I'm mulling over a couple of design considerations for allocators, and 
 was thinking of the following restriction:
 
 1. No out-of-bounds tricks and no pointer arithmetic. Consider:
 
 int[] a = new int[1000];
 int* p = a[500 .. $].ptr;
 a = a[250 .. 750];
 
 Subsequently the GC should be within its rights to deallocate any memory 
 within the first and last 250 integers allocated, even though in theory 
 the user may get to them by using pointer arithmetic.

I see that you are trying to account for allocator designs
that could reuse these memory fragments.
If this is for @safe, then maybe some memory could be released,
but you'd have to statically verify that internal pointers
don't make it into unsafe code where I wonder if any memory
would be released if I wrote:

size_t length = 100;
int* p = (new int[](length)).ptr;
GC.collect();
p[length-1] = 42;

So it is difficult to give a good answer. I'd say no until it
is clear how it would work outside of @safe.

 6. The point above brings to mind more radical possibilities, such as 
 making all arrays reference-counted and allowing compulsive deallocation 
 when the reference counter goes down to zero. That would rule out things 
 like escaping pointers to data inside arrays, which is quite extreme.

Would that affect all arrays, only arrays containing structs
or only affect arrays containing structs with dtors?

printf(hello\n.ptr);

should still work after all.

-- 
Marco

Re: FYI - mo' work on std.allocator

Am Thu, 01 May 2014 08:01:43 -0700
schrieb Andrei Alexandrescu seewebsiteforem...@erdani.org:

 On 5/1/14, 3:36 AM, Temtaime wrote:
  Hi Andrey. Have you even test your allocator on different arch(32/64)
  and/or with different compiler flags ?
 
 Thanks, I'll look into that! -- Andrei

If size_t was a distinct type ... just thinking loudly.

-- 
Marco

Re: FYI - mo' work on std.allocator