Re: New DConf Blog Post

2019-03-22 Thread Steven Schveighoffer via Digitalmars-d-announce

On 3/22/19 9:58 AM, Mike Parker wrote:
The DConf schedule was announced last Sunday. I've just published a 
write-up about it on the blog for the world-at-large. Please help us out 
by sharing this post in your social media circles.


The blog:
https://dlang.org/blog/2019/03/22/dconf-2019-london-programme/

Reddit:
https://www.reddit.com/r/programming/comments/b45bxp/dconf_2019_london_programme/ 



Side note, I started using mewe (and I really am not a social media 
type, so I doubt it will get a lot of use from me). And I created a 
Dlang group, if anyone is on there, feel free to join.


https://mewe.com/group/5c7174c35ff915267872a688

-Steve


Re: DConf 2019 Schedule

2019-03-19 Thread Steven Schveighoffer via Digitalmars-d-announce

On 3/19/19 10:24 AM, Mike Parker wrote:

On Tuesday, 19 March 2019 at 13:44:03 UTC, Steven Schveighoffer wrote:

On 3/19/19 9:27 AM, XavierAP wrote:

On Sunday, 17 March 2019 at 22:43:28 UTC, Mike Parker wrote:


http://dconf.org/2019/schedule/index.html


I see transcoding errors in several of the event pages, on Firefox. 
Am I the only one?


« The D version of the C++ copy constructor, historically called 
“postblit†»


« Sebastian has contributed to D’s core repositories »


No, you are not. Something happened, and the CSS is in chaos right 
now. Nothing looks good.




There was a missing close quote on the lang directive. It rendered fine 
locally, but not on the site. That's fixed now. Is that the only issue?


The other thing I noticed, look at the menu of a talk page: 
https://imgur.com/a/xprJCot


It just looks so bad that I thought there were more than one problem, 
but maybe that's the only remaining issue.


-Steve


Re: DIP 1016--ref T accepts r-values--Formal Assessment

2019-01-31 Thread Steven Schveighoffer via Digitalmars-d-announce

On 1/31/19 4:46 PM, Andrei Alexandrescu wrote:

On 1/31/19 4:42 PM, Olivier FAURE wrote:

On Thursday, 31 January 2019 at 16:38:42 UTC, Steven Schveighoffer wrote:

Yeah, that's already a thing that ref in D doesn't protect against:


It took me a while to understand what the compiler was doing.

This really feels like something that shouldn't compile.


The proposal could actually disallow rvalues that have lvalue syntax, 
such as "symbol", "symbol[expr]", "symbol.symbol", 
"symbol.symbol[expr]", etc. Ugh. Gets hairy quickly.


No, because those calls might actually do something with side effects! 
rvalues can contain other references.


The only way to fix this would be to overload member functions on the 
lvalue-ness of `this`. I don't recommend this at all, as I see this as a 
weird but rare problem that doesn't affect most D code.


-Steve


Re: DIP 1016--ref T accepts r-values--Formal Assessment

2019-01-31 Thread Steven Schveighoffer via Digitalmars-d-announce

On 1/31/19 4:42 PM, Olivier FAURE wrote:

On Thursday, 31 January 2019 at 16:38:42 UTC, Steven Schveighoffer wrote:

Yeah, that's already a thing that ref in D doesn't protect against:


It took me a while to understand what the compiler was doing.

This really feels like something that shouldn't compile.


The problem is that `this` is passed by reference EVEN for rvalues. 
Knowing this, you can construct difficult situations, but normally these 
don't appear in the wild.


If you've ever tried to make a simple math wrapper type, you would see 
how this is weird. And it has been like this since the beginning of D2.


You get things like:

a + Foo(1); // error
Foo(1) + a; // OK!

That being said, you can look at the fact that most people don't even 
know about this problem, even seasoned veterans, as a sign that it's 
really not a big problem.


-Steve


Re: DIP 1016--ref T accepts r-values--Formal Assessment

2019-01-31 Thread Steven Schveighoffer via Digitalmars-d-announce

On 1/31/19 4:46 PM, Olivier FAURE wrote:

On Thursday, 31 January 2019 at 18:31:22 UTC, Steven Schveighoffer wrote:

BTW, the DIP discusses how to annotate these rare situations:

int doubleMyValue(ref int x) { ... }
@disable int doubleMyValue(int x);



I don't think that's a solution. The problem is in the getter method, 
not in doubleMyValue. If nothing else, since the DIP is designed to work 
on existing functions, it could happen on doubleMyValue functions which 
would be both designed by and used by people completely unaware of 
DIP-1016.


How is the problem not in doubleMyValue? It's sole purpose is to update 
an lvalue. It is the perfect candidate to mark with @disable for rvalues.


-Steve


Re: DIP 1016--ref T accepts r-values--Formal Assessment

2019-01-31 Thread Steven Schveighoffer via Digitalmars-d-announce

On 1/31/19 11:04 AM, Olivier FAURE wrote:

On Thursday, 31 January 2019 at 02:10:05 UTC, Manu wrote:

I still can't see a truck-sized hole.


I don't know if it's truck-sized, but here's another corner case:

     int doubleMyValue(ref int x) {
     x *= 2;
     return x;
     }

     Point pt;
     pt.x = 5;
     pt.y = foobar();

     doubleMyValue(pt.x);
     assert(pt.x == 10);

Question: in the above code, will the assertion pass?

Answer: it depends on Point's implementation. If x is a member variable, 
then yes. If it's a getter, then doubleMyValue will take a rvalue and x 
won't be mutated and the assertion will fail.


I think this is a non-trivial conceptual problem.


BTW, the DIP discusses how to annotate these rare situations:

int doubleMyValue(ref int x) { ... }
@disable int doubleMyValue(int x);

-Steve


Re: DIP 1016--ref T accepts r-values--Formal Assessment

2019-01-31 Thread Steven Schveighoffer via Digitalmars-d-announce

On 1/31/19 11:04 AM, Olivier FAURE wrote:

On Thursday, 31 January 2019 at 02:10:05 UTC, Manu wrote:

I still can't see a truck-sized hole.


I don't know if it's truck-sized, but here's another corner case:

     int doubleMyValue(ref int x) {
     x *= 2;
     return x;
     }

     Point pt;
     pt.x = 5;
     pt.y = foobar();

     doubleMyValue(pt.x);
     assert(pt.x == 10);

Question: in the above code, will the assertion pass?

Answer: it depends on Point's implementation. If x is a member variable, 
then yes. If it's a getter, then doubleMyValue will take a rvalue and x 
won't be mutated and the assertion will fail.


I think this is a non-trivial conceptual problem.


Yeah, that's already a thing that ref in D doesn't protect against:

struct Point
{
   private int _x, _y;
   ref int x() { return _x; }
   ref int y() { return _y; }
}

struct Rect
{
   private Point _origin, _lengths;
   Point origin() { return _origin; }
   Point lengths() { return _lengths; }
   void origin(Point p) { _origin = p; }
   void lengths(Point p) { _lengths = p; }
}

Rect r;
r.origin = Point(1, 2);
r.lengths = Point(5, 5);
doubleMyValue(r.lengths.x);
assert(r.lengths.x == 10); // fail

-Steve


Re: DIP 1016--ref T accepts r-values--Formal Assessment

2019-01-31 Thread Steven Schveighoffer via Digitalmars-d-announce

On 1/31/19 3:25 AM, Walter Bright wrote:
But the DIP says const ref is not required. Therefore, copying an lvalue 
to a temporary cannot be allowed, therefore implicit conversion of 
lvalues cannot happen.


The biggest reason I see to not worry about const is that we already 
don't for member functions. And the world hasn't ended (yet).




Then we're faced with the question of if implicit conversion of lvalues 
is not allowed, should implicit conversion of rvalues be allowed? I'm 
not so sure it should be. For one thing, a lot of people are confused 
about lvalues vs rvalues, and would find the difference in behavior 
puzzling. For another, it can complicate overloading rules. I'd say 
allowing the conversions needs a strong rationale.


We could certainly start out with no conversions allowed (except you 
MUST allow conversions of compile-time data -- e.g. literals and CTFE 
produced values -- that is very key), and then relax the rules later if 
that's important.


-Steve


Re: DIP 1016--ref T accepts r-values--Formal Assessment

2019-01-31 Thread Steven Schveighoffer via Digitalmars-d-announce

On 1/31/19 2:26 AM, Andrei Alexandrescu wrote:

The trouble is major.

Replace "if" with "while":

while (ref_fun(10)) { ... }
==>
{
   int __tmp = 10;
   while (ref_fun(__tmp)) { ... }
}

That means ref_fun is called with the same lvalue multiple times. In all 
likelihood this is not what you want!


Yes, the trouble specifically is loops. Because loops execute their 
internal expressions over and over again.


Unfortunately, this means lowering isn't possible. That is, lowering to 
something expressible in the normal language isn't possible.


However, we all know that loops are essentially "lowered" in the AST to 
simple ifs and gotos. We just need to operate at that level.


So for instance the above looks something like this in AST:

loop_continue:
if(ref_fun(10))
{
  ...
}
else
   goto loop_end;
goto loop_continue;
loop_end:

(I know I'm omitting a lot of extra stuff like scope cleanup, that is 
implied here, as I don't know the exact details).


What needs to happen is the temporary (with extra scope)is inserted 
between the loop start and the if statement:


loop_continue:
{
   int __tmp = 10;
   if(ref_fun(__tmp))
   {
  ...
   }
   else
  goto loop_end;
}
goto loop_continue;
loop_end:

A possible retort is: "Of course, while would not be lowered that way, 
but a slightly different way!" etc. The point is, ALL OF THAT must be in 
the DIP, not assumed obvious or clarified in informal discusson outside 
the DIP.


Again: please be thorough, state your assumptions, cover all cases.


Agree, this needs to handle all possible cases. Really I think loops are 
the only problems, foreach, for, and while/do..while


A possible way forward is inventing a new syntax to allow declarations 
in this space, and then lowering can happen. Something similar to 
if(auto x = ...)


-Steve


Re: DIP 1016--ref T accepts r-values--Formal Assessment

2019-01-30 Thread Steven Schveighoffer via Digitalmars-d-announce

On 1/30/19 10:05 PM, Manu wrote:

On Wed, Jan 30, 2019 at 6:40 PM Nicholas Wilson via
Digitalmars-d-announce  wrote:

You should clarify that ;)


Yes, as said above, read `short(10)`. I can understand the confusion
that it may look like a variable when taken out of context; but listed
beneath the heading immediately above which says:
"This inconvenience extends broadly to every manner of **rvalue**
passed to functions"
It didn't occur to me the reader might interpret the clearly stated
list of cases of rvalues passed to functions to include arguments that
are not rvalues.
The name was just chosen to indicate the argument is a short, perhaps
an enum, or any expression that is a short... I could have used
`short(10)`, but apparently I didn't think of it at the time.

Is this the basis for the claims of "a hole you could drive a truck
through"? Again, a request for clarification, and a
couldn't-possibly-be-more-trivial revision may resolve this.



I think changing it to `short(10)` helps the argument that you didn't 
intend it to mean conversions from lvalues, but I'd recommend still 
spelling out that they are forbidden.


Leaving the reader to infer intent is not as good as clarifying intent 
directly. The whole rvalue vs. lvalue thing is confusing to me, because 
I assumed an lvalue converted to a different type changes it to an 
rvalue. I think of it like an implicit function that returns that new value.


-Steve


Re: DIP 1016 should use expression lowering, not statement lowering

2019-01-30 Thread Steven Schveighoffer via Digitalmars-d-announce

On 1/30/19 10:03 PM, Nicholas Wilson wrote:

On Thursday, 31 January 2019 at 02:29:47 UTC, Steven Schveighoffer wrote:

I came up with this idea based on tempCString, but it doesn't work:

So I don't get why it doesn't work. But if that was fixed, could be a 
potential workaround without requiring a DIP.


Thats nice! But it doesn't fix the problem that in generic code you 
don't know without checking if you need to do that. Also the template 
bloat.


Yeah, it could do this too:

auto ref rv(T)(auto ref T t)
{
   static if(__traits(isRef, t)) return t;
   else /* do the other stuff */
}

But yes, template bloat. Plus having to put rv on everything... would 
suck. The DIP to me seems like it should be good with the clarification 
of not applying to lvalue -> rvalue auto translations.


-Steve


Re: DIP 1016 should use expression lowering, not statement lowering

2019-01-30 Thread Steven Schveighoffer via Digitalmars-d-announce

On 1/29/19 6:52 AM, Andrei Alexandrescu wrote:

While writing this example:

int[] a = cast(int[]) alloc.allocate(100 * int.sizeof);
if (alloc.reallocate(a, 200 * int.sizeof))
{
     assert(a.length == 200);
}

=>

int[] a = cast(int[]) alloc.allocate(100 * int.sizeof);
void[] __temp0 = a;
if (alloc.reallocate(__temp0, 200 * int.sizeof)
{
     assert(a.length == 200);
}

I noticed a problem - the lowering as informally described in DIP 1016 
makes it difficult to figure how function calls present in control 
statements like if, while, etc. should behave. Where should the 
temporary go? An expression-based lowering clarifies everything. A 
statement-based lowering would need to work on a case basis for all 
statements involving expressions.


I don't think it's very difficult or confusing. Any rvalue in any 
expression that requires ref should be implicitly declared before the 
statement that contains the expression, and include a new scope:




>

{
   auto _tmpr1 = r1;
   auto _tmpr2 = r2;
   auto _tmpr3 = r3;
   ...
   auto _tmprN = rN;
   
}

Essentially, nothing is different from existing semantics today, when 
rvalues are used and provides reference semantics (yes, it's possible, 
see tempCString). They live until the end of the statement. It's how 
this has to be. It can't be expression based.


-Steve


Re: DIP 1016--ref T accepts r-values--Formal Assessment

2019-01-28 Thread Steven Schveighoffer via Digitalmars-d-announce

On 1/28/19 9:15 PM, Andrei Alexandrescu wrote:

On 1/28/19 5:23 PM, Steven Schveighoffer wrote:

I already see this kind of bug all the time with alias this.


Can you please post more detail? It may be of relevance to future work.


Any time you have the alias this, then you can get confused when calling 
a function expecting it to be sent as the original type, but the alias 
this is used instead. In cases where both are seemingly accepted by 
overloads, then it can be confusing which overload is used. Sometimes 
it's as simple as the overload that you were expecting to be called 
won't compile or is deselected by constraints. But the code still 
compiles and runs, just does something different from what you expected.


I meant that the effect is similar to what you were noting.

-Steve


Re: DIP 1016--ref T accepts r-values--Formal Assessment

2019-01-28 Thread Steven Schveighoffer via Digitalmars-d-announce

On 1/28/19 2:58 PM, Andrei Alexandrescu wrote:
TL;DR: it could be argued that the only dangerous conversions are lvalue 
-> temp rvalue -> ref, so only disable those. The conversion rvalue -> 
temp rvalue -> ref is not dangerous because the starting value on the 
caller side could not be inspected after the call anyway.


I agree with you. It's one thing for the caller to pass in an rvalue 
that can clearly no longer be accessed. It's another thing to pass in an 
lvalue and have it "update" a temporary instead.


I already see this kind of bug all the time with alias this.

-Steve


Re: DIP 1016--ref T accepts r-values--Formal Assessment

2019-01-25 Thread Steven Schveighoffer via Digitalmars-d-announce

On 1/25/19 5:57 AM, kinke wrote:

On Thursday, 24 January 2019 at 23:59:30 UTC, Walter Bright wrote:

On 1/24/2019 1:03 PM, kinke wrote:
(bool __gate = false;) , ((A __pfx = a();)) , ((B __pfy = b();)) , 
__gate = true , f(__pfx, __pfy);


There must be an individual gate for each of __pfx and pfy. With the 
rewrite above, if b() throws then _pfx won't be destructed.


There is no individual gate, there's just one to rule the 
caller-destruction of all temporaries. That's the current state, and 
there's no need for that to change. I was trying to say that a rewrite 
as expression, as requested as part of the assessment, clearly isn't 
enough, as the dtor expressions aren't visible this way, and neither is 
the scoping (when the dtor expression of `__pfx` comes into play etc.).


I think the point of the DIP is not to lower expressions. It makes no 
sense to, they have to be statements (just like all temporaries live 
until the end of a statement).


-Steve


Re: D-lighted, I'm Sure

2019-01-18 Thread Steven Schveighoffer via Digitalmars-d-announce

On 1/18/19 11:42 AM, Ron Tarrant wrote:

On Friday, 18 January 2019 at 15:08:48 UTC, Steven Schveighoffer wrote:

Nice read! And welcome to Ron! I too, started with BASIC, but on a 
Commodore 64 :)



Thanks, Steve.

Just to set the record straight, I only had access to that Coleco Adam 
for the few weeks I was in that Newfoundland outport. Within a year, I 
too had my very own C-64 plugged into a monster Zenith console job. 
Remember those? I don't remember what I paid for a used C-64, but the 
Zenith 26" was $5 at a garage sale up the street and another $5 for 
delivery.


I had to use my parents' TV in the living room :) And I was made to 
learn typing before I could play games on it, so cruel...


-Steve


Re: D-lighted, I'm Sure

2019-01-18 Thread Steven Schveighoffer via Digitalmars-d-announce

On 1/18/19 9:29 AM, Mike Parker wrote:
Not long ago, in my retrospective on the D Blog in 2018, I invited folks 
to write about their first impressions of D. Ron Tarrant, who you may 
have seen in the Lear forum, answered the call. The result is the latest 
post on the blog, the first guest post of 2019. Thanks, Ron!


As a reminder, I'm still looking for new-user impressions and guest 
posts on any D-related topic. Please contact me if you're interested. 
And don't forget, there's a bounty for guest posts, so you can make a 
bit of extra cash in the process.


The blog:
https://dlang.org/blog/2019/01/18/d-lighted-im-sure/

Reddit:
https://www.reddit.com/r/programming/comments/ahawhz/dlighted_im_sure_the_first_two_months_with_d/ 



Nice read! And welcome to Ron! I too, started with BASIC, but on a 
Commodore 64 :)


-Steve


Re: B Revzin - if const expr isn't broken (was Re: My Meeting C++ Keynote video is now available)

2019-01-17 Thread Steven Schveighoffer via Digitalmars-d-announce

On 1/17/19 2:31 PM, H. S. Teoh wrote:

On Thu, Jan 17, 2019 at 06:03:07PM +, Paul Backus via 
Digitalmars-d-announce wrote:
[...]

[2]
https://bartoszmilewski.com/2009/10/21/what-does-haskell-have-to-do-with-c/

[...]

Haha, seems D did better than C++ in this respect, but not quite at the
level of Haskell.

The C++ example of a template that takes templates and arguments and
declares another template is a perfect example of why C++ template
syntax is utterly horrible for doing these sorts of things.

Coming back to the D example at the end, I totally agree with the
sentiment that D templates, in spite of their significant improvements
over C++ syntax, ultimately still follow the same recursive model. Yes,
you can use CTFE to achieve the same thing at runtime, but it's not the
same thing, and CTFE cannot manipulate template argument lists (aka
AliasSeq aka whatever it is you call them).  This lack of symmetry
percolates down the entire template system, leading to the necessity of
the hack that Bartosz refers to.

Had template argument lists / AliasSeq been symmetric w.r.t. runtime
list manipulation, we would've been able to write a foreach loop that
manipulates the AliasSeq in the most readable way without needing to
resort to hacks or recursive templates.


well, there was no static foreach for that article (which I admit I 
didn't read, but I know what you mean).


But it's DEFINITELY not as easy as it could be:

import std.conv;

alias AliasSeq(P...) = P;

template staticMap(alias Transform, Params...)
{
alias seq0 = Transform!(Params[0]);
static foreach(i; 1 .. Params.length)
{
   mixin("alias seq" ~ i.to!string ~ " = AliasSeq!(seq" ~ 
(i-1).to!string ~ ", Transform!(Params[" ~ i.to!string ~ "]));");

}
mixin("alias staticMap = seq" ~ (Params.length-1).to!string ~ ";");
}

alias Constify(T) = const(T);
void main()
{
alias someTypes = AliasSeq!(int, char, bool);
pragma(msg, staticMap!(Constify, someTypes)); // (const(int), 
const(char), const(bool))

}

Note, that this would be a LOT easier with string interpolation...

mixin("alias seq${i} = AliasSeq!(seq${i-1}, 
Transform!(Params[${i}]));".text);


-Steve


Re: My Meeting C++ Keynote video is now available

2019-01-16 Thread Steven Schveighoffer via Digitalmars-d-announce

On 1/16/19 11:43 AM, Paolo Invernizzi wrote:

On Wednesday, 16 January 2019 at 16:30:17 UTC, Steven Schveighoffer wrote:

On 1/16/19 10:06 AM, Paolo Invernizzi wrote:


I'm waiting, for example, for a revamp of IO, just to start...


We're working on it...

https://github.com/schveiguy/iopipe
https://github.com/MartinNowak/io



I was exactly referring to that two... they are great, and I've used both!
Despite that, they seem stalled: any plan to go ahead in the medium term?


Slowly... Sorry, it's not my day job to improve these :)

But I am working currently on an http client using iopipe 
(https://github.com/schveiguy/httpiopipe) and improving/releasing the 
json parser that I wrote for my talk 2 years ago 
(https://github.com/schveiguy/jsoniopipe).


Hint: I hope to have a working REST client soon.

I also have a library that's stalled, but I have to pick it up again in 
order to deal with parsing using ranges (not yet published).


I think in order to make this more palatable, though, we really do need 
to focus on the io base with some sort of async fiber-based driver.



Thanks for your work (and to Martin too)


You're welcome! Thanks for the kind words. It's actually one of the more 
fun projects I have worked on.


-Steve


Re: My Meeting C++ Keynote video is now available

2019-01-16 Thread Steven Schveighoffer via Digitalmars-d-announce

On 1/16/19 10:06 AM, Paolo Invernizzi wrote:


I'm waiting, for example, for a revamp of IO, just to start...


We're working on it...

https://github.com/schveiguy/iopipe
https://github.com/MartinNowak/io

-Steve


Re: My Meeting C++ Keynote video is now available

2019-01-16 Thread Steven Schveighoffer via Digitalmars-d-announce

On 1/15/19 4:37 AM, Martin Tschierschke wrote:

On Monday, 14 January 2019 at 18:52:02 UTC, Jacob Carlborg wrote:

On 2019-01-14 15:42, Steven Schveighoffer wrote:

That's a bad example :) The clear answer is mysql-native, which is 
what vibe.d recommends.


Exactly, and I don't need five minutes for that. Five seconds is 
enough :)
Ok, bad example,  but let's say you want ORM mapping, too and to have 
the ability to switch to Postgres later? So what would you recommend?


I should have said that your point is mostly correct, just that this is 
a bad example :)


I've looked for ORM on code.dlang.org, and never found one yet that I 
liked. So the answer would take infinite seconds to complete.


But let's say we put ORM into Phobos. Certainly, it would get heavy use, 
but I would be likely to believe that it would not cover the problem 
space completely, and leave quite a bit to be desired.


I think that there are some things that can go into the standard library 
and live there happily. There are other things that should be left to 
the community, even though they are essential, simply because locking 
into one implementation/API/idea is too restrictive.


-Steve


Re: My Meeting C++ Keynote video is now available

2019-01-14 Thread Steven Schveighoffer via Digitalmars-d-announce

On 1/14/19 10:57 AM, Adam D. Ruppe wrote:

On Monday, 14 January 2019 at 14:56:00 UTC, bachmeier wrote:
Only a small sliver of programming involves anything where "overhead 
of a runtime" is an issue. I hope you intend this comment as 
pertaining to Better C usage.


Real D is the true better C. These improvements can improve in various 
situations.


That said though, I'd be against removing built-in classes and 
interfaces. They are useful in a lot of places built in... though I 
kinda wish the runtime code was a bit thinner and lighter too.


Some of the old crufty features of classes can be jettisoned like 
toString and toHash, and object.factory.


-Steve


Re: My Meeting C++ Keynote video is now available

2019-01-14 Thread Steven Schveighoffer via Digitalmars-d-announce

On 1/14/19 5:18 AM, Martin Tschierschke wrote:

On Monday, 14 January 2019 at 07:50:32 UTC, Walter Bright wrote:

On 1/13/2019 9:31 PM, Paul Backus wrote:
Scheme is probably the language that takes this idea of a minimal 
"core language" with powerful metaprogramming facilities the 
furthest, and the result is a fragmented ecosystem that makes writing 
portable, non-trivial programs close to impossible. (See "The Lisp 
Curse" [1].)


When something like an object system is made part of the language (or 
at the very least, the standard library), it becomes a focal point 
[2] that the community can coordinate around. Due to the diverse, 
distributed nature of any programming-language community, trying to 
coordinate through explicit communication is not really a viable 
option, so having these kinds of focal points is very important if we 
want to be able to work together on anything.


[1] http://winestockwebdesign.com/Essays/Lisp_Curse.html
[2] https://en.wikipedia.org/wiki/Focal_point_(game_theory)


Interesting cites, which provide a basis for why I've opposed AST 
macros, and why Ddoc and unittest are builtin (and a few other things).


Also, before std::string came along in C++, everyone invented their 
own string class, and as a result, nobody could share code.


This is exactly the argument to get a database driver 
(mysql,postgres...) and probably a webserver in std.


I don't like the idea of it, because there are so many approaches. Even 
different approaches among one server protocol.




But to avoid getting std.lib to big, the D Foundation might adopt some 
third party libs as core libraries, so they get maintained within the D 
Foundation Git account to make them somehow official.


We are now approaching the 1500 Dub package, and the ecosystem becomes 
more and more complex.


This is true, I searched yesterday for a decimal package that fits my 
use case (I'm getting decimal numbers as strings in a JSON library, with 
no limits as to what number of decimal places are supported). There were 
probably 4 or 5 that implement the general concept of a decimal type, 
but I was faced with a couple issues:


1. Is it maintained? Some of them are really old, some are old-ish, but 
could potentially be pretty stable
2. Does it have all the features I want? I am hesitant at this point to 
select something that has overflow problems, because I don't actually 
know how much would be too much.


With one central package, I can be sure that 1 is answered, but then I 
don't actually have any choices for whether it works for me or not. If 
it doesn't implement the features I want (basically, I want completely 
arbitrary precision AND number of digits), then I still have to roll my own.


Some packages are hard to solve every, if not most, requirements. Really 
that should be the defining feature of whether it goes into the standard 
library.


(https://code.dlang.org/  1482 packages: The search for mysql now 
returns 23 packages. Please tell me which to use for the back end, of 
your own vibe.d app. I give you 5 minutes...:-)

Regards mt.


That's a bad example :) The clear answer is mysql-native, which is what 
vibe.d recommends.


-Steve


Re: The D Blog in 2018

2019-01-02 Thread Steven Schveighoffer via Digitalmars-d-announce

On 1/2/19 10:01 AM, Mike Parker wrote:

It's time for the annual D Blog retrospective. Including the stats.

The blog:
https://dlang.org/blog/2019/01/02/the-d-blog-in-2018/

Reddit:
https://www.reddit.com/r/d_language/comments/abu43a/the_d_blog_in_2018/

In a few days I'll be publishing a look back at some of the happenings 
around DLand at large in 2018, including status updates where 
appropriate. There's a DMD release to blog about in the interim!


"The top five posts of 2017"

Fix that ;)

-Steve


Re: DConf 2019: Shepherd's Pie Edition

2018-12-28 Thread Steven Schveighoffer via Digitalmars-d-announce

On 12/28/18 11:31 AM, Adam D. Ruppe wrote:

But I'm telling you, DConf can learn from this stuff. Joakim is doing 
the community a service by trying to get you all to try some changes. 
Even baby step compromises can yield results at low risk.


Note that the proposals always ask for format (talk, panel, contest, 
interpretive dance).


So you can always propose a presentation that takes on a format that you 
think will be more useful.


I think the biggest problem with Joakim's post was simply that it was 
phrased like `why do you want to have such a stupid ritual, when your 
money can be better spent doing something else?` Things went downhill 
from there.


-Steve


Re: DConf 2019: Shepherd's Pie Edition

2018-12-24 Thread Steven Schveighoffer via Digitalmars-d-announce

On 12/24/18 2:44 AM, Joakim wrote:

On Sunday, 23 December 2018 at 22:36:05 UTC, Steven Schveighoffer wrote:


Huh? It's their decision, not yours. Even if the decision has no 
reason at all, it's still theirs. What is the problem? Start your own 
D "conference competitor" if you think you can do better.


They are accountable to the community, so the decision and its reasons 
matter.


My impression is that the community likes and benefits from these 
conferences, so everything's cool there.


I, for one, will not be donating to the foundation as long as 
they continue to waste money this way, just as others have said they 
won't donate as long as it doesn't put out a Vision document anymore or 
otherwise communicate what it's doing with their money.


Nobody is asking for your money for this conference (unless you want to 
attend), and if you feel this way, that's totally your choice. I like 
the results that come from the conferences, I've been to all of them 
since 2013, on my dime for 3, and with assistance for 3. I felt it was 
100% worth it for all.


Nobody cares to debate something that has already been scheduled and 
planned, the time to bring up concerns was earlier, when you brought 
it up before. But that failed to convince, now it's decided, time to 
move on.


So you agree with me that there's no point in "debating" it again, 
perhaps you should have addressed this comment to Mike then?


Mike didn't start the debate in this thread, you did. Consider how one 
feels when careful deliberation is made, and a final decision, combined 
with an announcement is made. Would you like to have people question 
your decisions AFTER they are made, and commitments have already been 
established? The time to question them is before they are made, not 
after. Questioning after is simply viewed (rightly) as sour grapes. You 
didn't get your way, move on.


If it's such a great idea, that should be an easy case to make, 
compared to the alternatives given. Yet all I get is a bunch of 
stone-walling, suggesting no reasoning was actually involved, just 
blindly aping others and the past.


It is easy, for those who have attended conferences and like them -- 
they work well. All past dconfs are shining examples. Just drop it and 
move on to something else. You lost the battle for this one, it's no 
longer up for discussion.


Heh, there was no "battle," as most of those responding didn't even 
understand what I wrote, like Iain above, gave no arguments (we "like 
them -- they work well"), and as finally clear from Mike and Walter's 
responses here, there was no real deliberation on the matter.


You think they just flipped a coin one day, and didn't think about any 
past experience at all? No real thinking must have gone into it because 
only intelligent people can come to the conclusion you reached, right? 
This kind of "debate" where the assumption is that only my way is 
correct is common out there these days, it's tiring. The best thing you 
can do is start a competing conference style and show how it works 
better. I'm sure Walter and Andrei would not discourage more D 
conferences or conference-like gatherings.


Since they don't take DConf seriously, I see no reason to either: I'll 
just start ignoring it from now on.


That's unfortunate, but not anything I can change. You have contributed 
a lot in terms of the android port, although I haven't really programmed 
in android (I have a tiny bit, with Xamarin (hated it) and a bit with 
Java (was OK, but crazy complicated) ). I hope at some point you 
reconsider, I'd love to see a presentation on it.


-Steve


Re: DConf 2019: Shepherd's Pie Edition

2018-12-23 Thread Steven Schveighoffer via Digitalmars-d-announce

On 12/22/18 12:22 PM, Joakim wrote:

On Saturday, 22 December 2018 at 17:13:06 UTC, Mike Parker wrote:

On Saturday, 22 December 2018 at 16:57:10 UTC, Joakim wrote:

I'm not trying to discuss it with you or the community. I'm asking 
the D team who're making this decision why it's being made, despite 
all the reasoning in that thread, and reiterating that it's a bad 
move. I suspect they're not thinking this through, but they can speak 
for themselves.


The decision was made because your reasoning failed to convince anyone 
involved in the planning that maintaining the current format of DConf 
is a mistake. Nor do they agree with you that it's a bad move. We like 
the current format and see no need to change it at this time.


I see, so you admit no reasoning was involved on your part? Because you 
present none, either there or here.


Huh? It's their decision, not yours. Even if the decision has no reason 
at all, it's still theirs. What is the problem? Start your own D 
"conference competitor" if you think you can do better.




If you would like to carry on another debate about this, please open 
another thread in thhe General forum. This one isn't the place for it. 
Thanks!


As I just noted, I don't care to "debate" it with people who make no 
arguments. Instead, I'm asking you or whoever made this horrible 
decision why it's being made.


Nobody cares to debate something that has already been scheduled and 
planned, the time to bring up concerns was earlier, when you brought it 
up before. But that failed to convince, now it's decided, time to move on.


If it's such a great idea, that should be an easy case to make, compared 
to the alternatives given. Yet all I get is a bunch of stone-walling, 
suggesting no reasoning was actually involved, just blindly aping others 
and the past.


It is easy, for those who have attended conferences and like them -- 
they work well. All past dconfs are shining examples. Just drop it and 
move on to something else. You lost the battle for this one, it's no 
longer up for discussion.


-Steve


Re: Blog post: What D got wrong

2018-12-20 Thread Steven Schveighoffer via Digitalmars-d-announce

On 12/19/18 2:58 PM, Neia Neutuladh wrote:

On Wed, 19 Dec 2018 17:28:01 +, Vijay Nayar wrote:

Could you please elaborate a little bit more on this?  In the linked
program, I had expected that "ref" would return a reference to "a" that
would behave similar to a pointer.


They work like pointers that automatically dereference when assigning to
the base type.

Only three things in D can be ref:
* A function parameter
* A function return value
* A foreach variable (since that's either going to be a function return
value, a function parameter, or a pointer, depending on what you're
iterating over)

So when the compiler sees something like:

 ref int foo();
 auto a = foo();

It sees that the type of 'a' has to be the same as the return type of
'foo'. Except that's not possible, so it uses the nearest equivalent type:
int.


I would say it a little bit differently -- the return *type* of foo is 
int. In D, ref is not part of the type at all.


For example, even when it *could* use ref, it doesn't:

int x;
ref int foo() { return x; }

void bar(Args...)(Args args)
{
   pragma(msg, __traits(isRef, args[0]));
}

bar(foo()); // outputs false at compile time

The storage class is completely separate from the type. Which is 
different from C++, where it's like a storage class but is really a type 
constructor, hence Walter's post.


-Steve


Re: Announcing Elembuf

2018-12-19 Thread Steven Schveighoffer via Digitalmars-d-announce

On 12/19/18 12:54 PM, H. S. Teoh wrote:

On Wed, Dec 19, 2018 at 11:56:44AM -0500, Steven Schveighoffer via 
Digitalmars-d-announce wrote:

I had expected *some* improvement, I even wrote a "grep-like" example
that tries to keep a lot of data in the buffer such that moving the
data will be an expensive copy. I got no measurable difference.

I would suspect due to that experience that any gains made in not
copying would be dwarfed by the performance of network i/o vs. disk
i/o.

[...]

Ahh, that makes sense.  Did you test async I/O?  Not that I expect any
difference there either if you're I/O-bound; but reducing CPU load in
that case frees it up for other tasks.  I don't know how easy it would
be to test this, but I'm curious about what results you might get if you
had a compute-intensive background task that you run while waiting for
async I/O, then measure how much of the computation went through while
running the grep-like part of the code with either the circular buffer
or the moving buffer when each async request comes back.

Though that seems like a rather contrived example, since normally you'd
just spawn a different thread and let the OS handle the async for you.


The expectation in iopipe is that async i/o will be done a-la vibe.d 
style fiber-based i/o.


But even then, the cost of copying doesn't go up -- if it's negligable 
in synchronous i/o, it's going to be negligible in async i/o. If 
anything, it's going to be even less noticeable. It was quite a 
disappointment to me, actually.


-Steve


Re: Announcing Elembuf

2018-12-19 Thread Steven Schveighoffer via Digitalmars-d-announce

On 12/18/18 8:41 PM, H. S. Teoh wrote:

On Tue, Dec 18, 2018 at 01:56:18PM -0500, Steven Schveighoffer via 
Digitalmars-d-announce wrote:

On 12/18/18 10:36 AM, H. S. Teoh wrote:

On Tue, Dec 18, 2018 at 08:00:48AM +, Cyroxin via Digitalmars-d-announce 
wrote:

[...] While the focus of this library is in socket receival,
reading from a file doesn't seem to be bad either.

[...]

Ahh, I see. I thought the intent was to read from a file locally. If
you're receiving data from a socket, having a circular buffer makes
a lot more sense.  Thanks for the clarification.  Of course, a
circular buffer works pretty well for reading local files too,
though I'd consider its primary intent would be better suited for
receiving data from the network.


Although I haven't tested with network sockets, the circular buffer I
implemented for iopipe
(http://schveiguy.github.io/iopipe/iopipe/buffer/RingBuffer.html)
didn't have any significant improvement over a buffer that moves the
data still in the buffer.

[...]

Interesting. I wonder why that is. Perhaps with today's CPU cache
hierarchies and read prediction, a lot of the cost of moving the data is
amortized away.


I had expected *some* improvement, I even wrote a "grep-like" example 
that tries to keep a lot of data in the buffer such that moving the data 
will be an expensive copy. I got no measurable difference.


I would suspect due to that experience that any gains made in not 
copying would be dwarfed by the performance of network i/o vs. disk i/o.


-Steve


Re: Announcing Elembuf

2018-12-18 Thread Steven Schveighoffer via Digitalmars-d-announce

On 12/18/18 10:36 AM, H. S. Teoh wrote:

On Tue, Dec 18, 2018 at 08:00:48AM +, Cyroxin via Digitalmars-d-announce 
wrote:

[...] While the focus of this library is in socket receival, reading
from a file doesn't seem to be bad either.

[...]

Ahh, I see. I thought the intent was to read from a file locally. If
you're receiving data from a socket, having a circular buffer makes a
lot more sense.  Thanks for the clarification.  Of course, a circular
buffer works pretty well for reading local files too, though I'd
consider its primary intent would be better suited for receiving data
from the network.


Although I haven't tested with network sockets, the circular buffer I 
implemented for iopipe 
(http://schveiguy.github.io/iopipe/iopipe/buffer/RingBuffer.html) didn't 
have any significant improvement over a buffer that moves the data still 
in the buffer.


-Steve


Re: Blog post: What D got wrong

2018-12-18 Thread Steven Schveighoffer via Digitalmars-d-announce

On 12/17/18 4:42 AM, Dukc wrote:

On Monday, 17 December 2018 at 09:41:01 UTC, Dukc wrote:

On Saturday, 15 December 2018 at 19:53:06 UTC, Atila Neves wrote:


@safe and pure though...


Why @safe? Can't you just write "@safe:" on top and switch to 
@system/@trusted as needed?


Argh, I forgot that you are not supposed to @safe templates away.


You can apply @safe to anything. It's @trusted you have to be careful with.

Of course, it probably won't work for many templates.

-Steve


Re: D is helping with porch pirates

2018-12-18 Thread Steven Schveighoffer via Digitalmars-d-announce

On 12/17/18 6:13 PM, Daniel Kozák wrote:

https://gma.abc/2zWvXCl




Fixed subject, nice story!

-Steve


Re: A brief survey of build tools, focused on D

2018-12-11 Thread Steven Schveighoffer via Digitalmars-d-announce

On 12/11/18 12:39 PM, H. S. Teoh wrote:

On Tue, Dec 11, 2018 at 09:58:39AM +, Atila Neves via 
Digitalmars-d-announce wrote:

On Monday, 10 December 2018 at 22:18:28 UTC, Neia Neutuladh wrote:

[...]

In typical D code, it's usually faster to compile per package than
either all-at-once or per module. Which is why it's the default in
reggae.


Yeah, for projects past a certain size, compiling per package makes the
most sense.


[...]

 From discussions on IRC about reducing compile times, though, using
Phobos is a good way to get slow compilation, and I use Phobos. That
alone means incremental builds are likely to go long.


Yes. Especially with -unittest.


We've talked about this before.  Jonathan Marler actually ran a test and
discovered that it wasn't something *directly* to do with unittests; the
performance hit was coming from some unexpected interactions with the
way the compiler instantiates templates when -unittest is enabled.  I
don't remember what the conclusion was, though.


I remember:

1. When unittests are enabled, -allinst is enabled as well.
2. This means that all templates instantiated are included as if they 
were part of the local module.
3. This means that they are semantically analyzed, and if they import 
anything, all those imports are processed as well

4. Recurse on step 2.

Note that the reason allinst is used is because sometimes templates 
compile differently when unittests are enabled. In other words, you 
might for instance get a different struct layout for when unittests are 
enabled -- this prevents that (but only for templates of course).


The ultimate reason why the PR (which removed the -allinst flag for 
unittests) was failing was because of differences in compiler flags for 
different modules during unittests in Phobos. This caused symbol name 
mangling changes (IIRC, mostly surrounding dip1000 problems).


I really wish we could have followed through on that PR...

-Steve


Re: Blog post: What D got wrong

2018-12-11 Thread Steven Schveighoffer via Digitalmars-d-announce

On 12/11/18 5:45 AM, Atila Neves wrote:

A few things that have annoyed me about writing D lately:

https://atilanevesoncode.wordpress.com/2018/12/11/what-d-got-wrong/


Agree with most of this. UFCS for templates would be awesome, but the 
syntax is trickier, since instantiation uses ! instead of .


I can't see how you get around the ambiguities, especially when a 
template could be a UFCS function.


inout: template This doesn't work the same (no guarantee it doesn't 
mutate the data). And it still generates multiple copies of the same 
function. As I said in my dconf 2016 presentation and in numerous 
debates on this forums, the only reason to care about const or inout is 
if you care about mutability guarantees on the callee. We can make do 
with templates and immutable without either of those features.


@property: This was almost about to be awesome, but squabbling amongst 
the D core team killed it. Now, it's only a very obscure difference:


struct S
{
   int _x;
   int x1() { return _x; }
   @property int x2() { return _x; }
}

pragma(msg, typeof(S.x1)); // int()
pragma(msg, typeof(S.x2)); // int


Which is COMPLETELY USELESS, since you have to handle both cases anyway.

-Steve


Re: Ported js13k game underrun to D targeting webassembly

2018-11-18 Thread Steven Schveighoffer via Digitalmars-d-announce

On 11/17/18 4:35 PM, Sebastiaan Koppe wrote:
Underrun is small game build by Dominic Szablewski for the 2018 
js13kGames competition.


I decided to port it to D and to target webassembly. You can play the 
game here https://skoppe.github.io/spasm/examples/underrun/


It is part of my endeavour into the wonderful world of webassembly. I 
specifically choose this project so I could get a feel for how to call 
web api's other than the dom, i.e. WebGL and AudioContext.


Porting went pretty smooth, except for the fact that I wanted it to be 
done in betterC, which turned out to cripple lots of things I am used to 
rely on.


Things like:

- having to write closures by hand
- not being able to use stdx.allocator
- almost no use of phobos (even at CTFE)

D could definitely use some love in that area. I was specifically 
stumped that stdx.allocator doesn't work in betterC; if somewhere is a 
good place to use stdx.allocator you would expect it to be betterC, but 
alas.


Had lots of fun though. Hope you like it.


Really cool. I didn't play the original, but I found this reasonably 
straightforward.


I love this this webassembly stuff, I think it's a fantastic 
demonstration, and test for the power of D.


A system to do closures by hand should be doable with templates I would 
think. Do you have a quick example?


-Steve


Re: DIP 1015--Deprecation of Implicit Conversion of Int. & Char. Literals to bool--Formal Assement

2018-11-15 Thread Steven Schveighoffer via Digitalmars-d-announce

On 11/14/18 4:32 PM, Neia Neutuladh wrote:

On Wed, 14 Nov 2018 13:40:46 -0500, Steven Schveighoffer wrote:

You don't think this is confusing?

enum A : int {
  val
}

A a;
foo(a); // error: be more specific
int x = a;
foo(x); // Sure


I find this confusing:

 void foo(int i) {}
 void foo(ubyte b) {}
 enum A : int { val = 0 }
 foo(A.val);  // calls foo(ubyte)
 A a = A.val;
 foo(a);  // calls foo(int)

If it instead produced an error, the error would look like:

 Error: foo called with argument types (E) matches both:
 example.d(1): foo(int i)
 and:
 example.d(2): foo(ubyte i)


I'm reminded of my son, who I sometimes give him a glass of water when 
he asks for it, and after I hand it to him, he says "should I drink this?"


To me, making the user jump through these hoops will be insanely 
frustrating.




Or else:

 Error: none of the overloads of foo are callable using
 argument types (A), candidates are:
 example.d(1): foo(int i)
 example.d(2): foo(ubyte i)

These aren't the intuitively obvious thing to me, but they're not going to
surprise me by calling the wrong function, and there are obvious ways to
make the code work as I want. Of the two, I'd prefer the former.


I prefer the correct version which calls foo(int). I think truly this is 
a misapplication of the overload rules, and can be fixed.



The intuitively obvious thing for me is:

* Don't use VRP to select an overload. Only use it if there's only one
candidate with the right number of arguments.
* Don't use VRP if the argument is a ctor, cast expression, or symbol
expression referring to a non-builtin. Maybe disallow with builtins.
* Don't use VRP if the argument is a literal with explicitly indicated type
(0UL shouldn't match to byte, for instance).


To me, an enum based on int is an int before it's a typeless integer 
value. VRP should be trumped by type (which it normally is). I would say 
an enum *derives* from int, and is a more specialized form of int. It is 
not derived from byte or ubyte. For it to match those overloads *over* 
int is surprising.


Just like you wouldn't expect the `alias this` inside a class to match 
an overload over its base class.


Oh, crap, it actually does... But I think that's a different bug.



I think this would make things more as most people expect:

 foo(A.val);  // A -> int, but no A -> byte; calls foo(int)
 foo(0);  // errors (currently calls foo(int))


If you have foo(int) and for some reason you can't call it with foo(0), 
nobody is going to expect or want that.



 foo(0L); // errors (currently calls foo(ubyte))
 foo(cast(ulong)0);  // errors (currently calls foo(ubyte))


I'm actually OK with this being the way it is, or even if it called the 
foo(int) version. Either way, as long as there is a clear definition of 
why it does that.




And when there's only one overload:

 void bar(byte b) {}
 bar(A.val);  // errors; can't convert A -> byte
 bar(0);  // type any-number and fits within byte, so should work
 bar(0UL);// errors; explicit incorrect type
 bar(0UL & 0x1F);// bitwise and expression can do VRP
 bar("foo".length);  // length is a builtin; maybe do VRP?
 bar(byte.sizeof);   // sizeof is a builtin; maybe do VRP?



I am OK with VRP calls, even in the case of the enum when there's no int 
overload.


-Steve


Re: DIP 1015--Deprecation of Implicit Conversion of Int. & Char. Literals to bool--Formal Assement

2018-11-14 Thread Steven Schveighoffer via Digitalmars-d-announce

On 11/14/18 1:11 PM, Neia Neutuladh wrote:

On Tue, 13 Nov 2018 20:27:05 -0800, Walter Bright wrote:

There have been various attempts over the years to "fix" various things
in the D matching system by adding "just one more" match level.


I kind of feel like, if something would be confusing like this, maybe the
compiler shouldn't be making an automatic decision. Not "just one more"
match level, but just...don't match. If there are multiple matching
overloads, just error out. Don't try to be clever and surprise people,
just tell the user to be more explicit.



You don't think this is confusing?

enum A : int
{
val
}

A a;
foo(a); // error: be more specific
int x = a;
foo(x); // Sure

-Steve


Re: DIP 1015--Deprecation of Implicit Conversion of Int. & Char. Literals to bool--Formal Assement

2018-11-13 Thread Steven Schveighoffer via Digitalmars-d-announce

On 11/13/18 11:26 AM, Chris M. wrote:

On Monday, 12 November 2018 at 09:45:14 UTC, Mike Parker wrote:
DIP 1015, "Deprecation and removal of implicit conversion from integer 
and character literals to bool, has been rejected, primarily on the 
grounds that it is factually incorrect in treating bool as a type 
distinct from other integral types.


The TL;DR is that the DIP is trying to change behavior that is working 
as intended.


From Example A in the DIP:

    bool b = 1;

This works because bool is a "small integral" with a range of 0..1. 
The current behavior is consistent with all other integrals.


From Example B in the DIP:

```
int f(bool b) { return 1; }
int f(int i) { return 2; }

enum E : int
{
    a = 0,
    b = 1,
    c = 2,
}
```

Here, f(a) and f(b) call the bool overload, while f(c) calls the int 
version. This works because D selects the overload with the tightest 
conversion. This behavior is consistent across all integral types. 
Replace bool with ubyte and f(a), f(b) would both call the ubyte 
version. The same holds for the DIP's Example C.


Walter and Andrei left the door open to change the overload behavior 
for *all* integral types, with the caveat that it's a huge hurdle for 
such a DIP to be accepted. It would need a compelling argument.


You can read a few more details in the summary I appended to the DIP:

https://github.com/dlang/DIPs/blob/master/DIPs/rejected/DIP1015.md#formal-assessment 



Thanks to Mike Franklin for sticking with the process to the end.


I was going to write something up about how you can't do arithmetic on 
bool types therefore they aren't integral, but I tested and realized D 
allows this (i.e. bool + bool, bool * bool). Still that seems 
nonsensical, so I guess my question what is the definition of an 
integral type and how does bool fit?


What it's doing is promoting false to an integer 0 and true to an 
integer 1. These work just like all the other integer promotion rules.


Interestingly enough, since bool can only be promoted to 0 or 1, bool * 
bool can be assigned back to a bool, but bool + bool can't be assigned 
back to a bool unless you cast. But don't expect the addition to behave 
as other integers do, as true + true == 2, which then casts to true.


-Steve


Re: DIP 1015--Deprecation of Implicit Conversion of Int. & Char. Literals to bool--Formal Assement

2018-11-13 Thread Steven Schveighoffer via Digitalmars-d-announce

On 11/12/18 4:38 PM, Walter Bright wrote:

On 11/12/2018 8:28 AM, 12345swordy wrote:
The issue that I see is unintended implicit conversation when passing 
values to functions that have both int and bool overloads.


The exact same thing happens when there are both int and short overloads.

The underlying issue is is bool a one bit integer type, or something 
special? D defines it as a one bit integer type, fitting it into the 
other integer types using exactly the same rules.


D's definition is wanting. Most integer types act differently than bool:

1. Integer types can be incremented, bool cannot
2. Integer types truncate by removing the extraneous bits, bool 
truncates to `true` for all values except 0.

3. Integer types have signed and unsigned variants, bool does not.
4. Integer types allow negation, bool does not.
5. Integer types can be used in a foreach(x; v1 .. v2), bool cannot.

It is true that bools act similarly to a 1-bit integer type in many 
cases, but only via promotion. That is, they *convert* to 1-bit 
integers, but don't behave like integers in their own type.


Regarding enums with base types, I admit I would totally expect an enum 
based on int to match an int overload over a short overload.


You don't think this is confusing to an average developer?

import std.stdio;

void foo(int x)
{
writeln("integer");
}

void foo(short x)
{
writeln("short");
}

enum A : int
{
a = 1,
b = 2,
c = 3
}

void main()
{
auto a = A.a;
foo(A.a); // case 1
foo(a);   // case 2
}

case 1 prints short, but case 2 prints integer. Both are passed the same 
value. This comes into play when using compile-time generation -- you 
are expecting the same behavior when using the same values. This is 
super-confusing.


But on the other hand, an A can ONLY be 3 or less, so why doesn't case 2 
print short? If VRP is used here, it seems lacking.


Maybe the biggest gripe here is that enums don't prefer their base types 
over what their base types convert to. In the developer's mind, the 
conversion is:


A => int => (via VRP) short

which seems more complex than just

A => int

If it is to be a special type with special rules, what about the other 
integer types? D has a lot of basic types :-)


The added value of having bool implicitly cast to integer types is 
great. I wouldn't want to eliminate that. The other way around seems of 
almost no value, except maybe to avoid extra code in the compiler.


So, I would be fine to have bool be a non-integer type that implicitly 
casts to integer for use in math or other reasons. But having 1 or 0 
implicitly cast to true or false has little value, and especially using 
it as "just another 1-bit integer", which it really isn't, has almost no 
usage.


-Steve


Re: DIP 1015--Deprecation of Implicit Conversion of Int. & Char. Literals to bool--Formal Assement

2018-11-12 Thread Steven Schveighoffer via Digitalmars-d-announce

On 11/12/18 4:45 AM, Mike Parker wrote:
DIP 1015, "Deprecation and removal of implicit conversion from integer 
and character literals to bool, has been rejected, primarily on the 
grounds that it is factually incorrect in treating bool as a type 
distinct from other integral types.


The TL;DR is that the DIP is trying to change behavior that is working 
as intended.


 From Example A in the DIP:

     bool b = 1;

This works because bool is a "small integral" with a range of 0..1. The 
current behavior is consistent with all other integrals.


But it's not consistent:

void isItAnInteger(T, bool makeCompile = false)() // works for all integers
{
   T val = T.min; // I was surprised these work for bool
   val = T.max;
   static if(!makeCompile)
   {
   long x = -val; // Error: operation not allowed on bool b
   ++val; // Error: operation not allowed on bool val += 1
   val += 1; // same error
   }
   val = cast(T)(T.max + 1);
   assert(val == val.min); // error for bool, true + 1 == 2, but 
cast(bool)2 truncates to true, not false.

}

void main()
{
import std.meta;
static foreach(T; AliasSeq!(int, uint, long, ulong, short, ushort, 
byte, ubyte))

{
isItAnInteger!T();
}

// switch second parameter to false to see compiler errors.
isItAnInteger!(bool, true)();
}

If you have the makeCompile flag set to true, then it asserts for bool, 
but nothing else.


-Steve


Re: Wed Oct 7 - Avoiding Code Smells by Walter Bright

2018-11-05 Thread Steven Schveighoffer via Digitalmars-d-announce

On 11/5/18 7:19 AM, Codifies wrote:
I subscribed to this forum in the hope I'd get irregular updates on 
useful and interesting things related to the D language.


This thread as far as I see it had degenerated into a somewhat childish 
and unproductive waste of time, I wouldn't object to a moderator locking 
this thread


There is a troll here posting as multiple different aliases, who has 
tried this before, and continually comes back to harp on the same issue. 
It's why I haven't participated, he doesn't need to have more encouragement.


Just give it time, he will give up and go back to being a lurker. It 
would be good if people just stop responding here.


-Steve


Re: Spasm - webassembly libary for single page applications

2018-10-12 Thread Steven Schveighoffer via Digitalmars-d-announce

On 10/12/18 3:43 PM, Sebastiaan Koppe wrote:

I like to announce Spasm https://github.com/skoppe/spasm

It is a webassembly library to develop single page applications and 
builds on my previous work 
(https://forum.dlang.org/post/eqneqstmwfzugymfe...@forum.dlang.org).


It generates fast and small webassembly binaries. The example todo-mvc 
is only 5995 bytes (wasm) + 2199 bytes (html+js) when gzipped, and loads 
pretty fast (https://skoppe.github.io/spasm/examples/todo-mvc/)


Spasm can be used with dub and only requires a recent version of ldc. 
See the readme on github how to get started.


The library includes a webpack bootstrap project to build a production 
ready web page.


Spasm is written in betterC.

Next steps are:

- more js host bindings (xhr, localstorage, etc.)
- memory deallocation
- gather an angry mob to make phobos more betterC compatible ;)


OK, this is pretty cool. I need to start checking out webasm.

-Steve


Re: fearless v0.0.1 - shared made easy (and @safe)

2018-09-19 Thread Steven Schveighoffer via Digitalmars-d-announce

On 9/19/18 6:44 AM, Atila Neves wrote:

On Tuesday, 18 September 2018 at 17:34:10 UTC, 12345swordy wrote:

On Tuesday, 18 September 2018 at 17:20:26 UTC, Atila Neves wrote:

The `shared` keyword currently means one of two things:

1. You can use core.atomic with it
2. It's some struct and you BYOM (Bring Your Own Mutex)

[...]


Why is this is an external 3rd party library isn't of the standard 
library?


Because it's far easier to release something with dub. If it gets 
traction then it might be a candidate for inclusion in Phobos.


I'll also add that if it depends on dip1000, Phobos isn't ready for that 
anyway.


But if Phobos does end up getting to that point, this 100% should be in 
the standard library. It's a no-brainer.


-Steve


Re: A facebook group for D programmers

2018-09-16 Thread Steven Schveighoffer via Digitalmars-d-announce

On 9/16/18 2:51 PM, Peter Alexander wrote:

On Sunday, 16 September 2018 at 20:19:32 UTC, Murilo wrote:
Hello everyone, I was so amazed with the D language that I created a 
facebook group for us all to be connected and share information. It is 
called "Programming in D", it has already 55 members. Please join the 
group and invite everyone else to join it. That way we can show the 
world how amazing the D language is.


Probably would be a good idea to link to the group. I couldn't find it 
with search.


This seems pretty... spamish.

Apologies if that's not true, but the original message is so 
fill-in-the-blank-with-target-topic that it's hard to take seriously. 
Also the "Already has 55 members" seems weird too. Especially if it's 
never been announced before.


-Steve


Re: [OT] My State is Illegally Preventing Me From Voting In The Upcoming 2018 US Elections

2018-09-15 Thread Steven Schveighoffer via Digitalmars-d-announce

On 9/9/18 2:34 AM, Nick Sabalausky (Abscissa) wrote:
[snip]

I personally think you are overreacting.

Have you voted before? If so, just go to the place you did before. It's 
all I ever do. If you haven't, and go to the wrong place, I'm sure they 
will help you find the right place. In my state (MA), it's usually a school.


We have signs up in our town weeks before the election reminding of the 
upcoming vote, and where it is. I seriously doubt that the ONLY way 
people can figure out their polling place is via the Internet. 
Especially if said Internet site is implemented by the government, which 
invariably will have stupid problems. And if there is some grand 
conspiracy to prevent people from voting by not letting them know their 
polling place, it's a pretty poorly designed scheme. Just ask your 
neighbor where to vote, I'm sure you'll figure it out.


I recently tried to pay a traffic ticket online (first I've had in like 
15 years), and there was a requirement to put in a code that the officer 
wrote down. The field was a drop-down and DIDN'T contain the code that 
the officer wrote. I called them, and they confirmed the code the 
officer wrote was correct. I tried editing the web page to include the 
code and submit, and it still failed.


I ended up having to mail it in.

-Steve


Re: silly is released - new test runner for the D programming language

2018-08-15 Thread Steven Schveighoffer via Digitalmars-d-announce

On 8/12/18 11:07 AM, Anton Fediushin wrote:

Hello, I'm glad to announce that silly v0.0.1 is released.

Silly is a brand-new test runner with simplicity in mind. It's developed 
to be as simple as possible and contain no useless features. Another 
important goal is to provide flexible tool which can be easily 
integrated into existing environments.


Instead of using dub as part of the test runner or hacking into 
`preBuildCommands` silly seamlessly integrates into `dub test` requiring 
user to just add it as a dependency into dub.json or dub.sdl.


Check out project's website [1], repository [2] and page on dub registry 
[3] where you can find more information.


[1]: https://ohboi.gitlab.io/silly/
[2]: https://gitlab.com/ohboi/silly
[3]: https://silly.dub.pm/



I notice that you're using the new unittest system added in 2.078, but 
bypassing the reporting mechanism (i.e. using exit instead of 
returning). Is there a reason why?


This is exactly the type of thing I envisioned would be used when 
developing said feature. I'm asking because I'd like to understand what 
the limitations are that caused you to bypass it.


-Steve


D Boston Meetup - 8/22

2018-08-08 Thread Steven Schveighoffer via Digitalmars-d-announce

Hi all,

We are meeting again at the Capital One Cafe [1] on August 22nd at 6pm 
for a meetup to continue (start?) progress on the NLP library and to 
converse more about our favorite language. Sameer and Michael are going 
to be there, as well as myself.


If you are in the Boston area around that time and want to hang out, 
drop on by! Respond here or send me an email to let me know you are coming.


-Steve

[1] 799 Boylston St, Boston, MA 02116


Re: On D in competitive programming

2018-07-30 Thread Steven Schveighoffer via Digitalmars-d-announce

On 7/28/18 3:51 PM, Ivan Kazmenko wrote:

Hey,

I wrote a post with my general reflections on using D in competitive 
programming.
Mostly compared to C++, since that's what more than 90% of people use 
for it.

The post is tailored to cover only the competitive programming specifics.

http://codeforces.com/blog/entry/60890
(en+ru, the language switch is at the top)



Good read.

a lifetime ago, I competed using topcoder (and wrote a bunch of problem 
sets for them too). Topcoder had a "challenge" phase, where you could 
challenge the solutions of others.


Is there anything like that in codeforces, and if so, is D an advantage 
as a "somewhat obscure" language (i.e. competitors can't always 
understand your code)?


Just curious :)

-Steve


Re: Hunt framework 1.2.0 released

2018-07-26 Thread Steven Schveighoffer via Digitalmars-d-announce

On 7/17/18 2:49 PM, Brian wrote:

On Tuesday, 17 July 2018 at 11:10:07 UTC, Suliman wrote:

On Tuesday, 17 July 2018 at 09:27:26 UTC, Brian wrote:

Hello, hunt framework fix bugs version release.

Major updates:
1. Add simplify functions
2. You can use createUrl() to create link url from 
module.controller.action

3. support date() / url() function for template engine
4. fix multi-domain use other port
5. use portgresql / mysql / sqlite on default config
6. upgrade dependencies package version to latest
7. release stabled skeleton project

hunt-framework source code:
https://github.com/huntlabs/hunt

hunt-skeleton source code:
https://github.com/huntlabs/hunt-skeleton


Could you explain what your vision about benefits Hunt over Vibed?


Vibe.d is very cool and fast network and data management library.

but, we need a web framework like spring boot, django, laravel, rails.

It's so easy to use, so simple to configuration and quickly for 
development.


My team develop the Entity(Like java's JPA & PHP's doctrine2), 
cache(Like ehcache), Collie(Like netty), entity-repository(Like 
spring-data) to development people are more familiar with the 
hunt-framework.


Very cool, I'll have to try this next web project. What is the io piece 
based on? I know that Diamond is based on vibe.d.  If home grown, what 
type of IO system is it?


-Steve


D Boston Meetup - 7/26

2018-07-10 Thread Steven Schveighoffer via Digitalmars-d-announce

Hi all,

We are going to set up a little coding session/gathering for D in the 
Capital One Cafe in Back bay [1] on July 26th (Thursday) starting at 6 
pm. So far it looks like Sameer, myself and Andrei will be there.


Please stop by if you want to hang out, chat about our favorite 
language, or any other reason! We will be working on porting NLP 
libraries, fixing dmd bugs (have one on my radar), or working on other 
projects.


-Steve

[1] 799 Boylston St, Boston, MA 02116


Re: I have a plan.. I really DO

2018-07-06 Thread Steven Schveighoffer via Digitalmars-d-announce

On 7/6/18 11:53 AM, Ecstatic Coder wrote:

Of course, the answer in C++ is that it won't compile, this is D code! ;)


Seriously ?


No, not seriously! I realized what you meant.


I wrote : "And what about the same code in C++ ?"

I thought people on this forum were smart enough to understand "the C++ 
port of this D code".


It was a point that we are delving into the absurd, saying "it depends 
on what type point_value is".



I'm sorry to have been wrong on this.


Sorry I made it seem like I was serious, my humor can be very dry sometimes.

Anyway, what nobody here *wants* to understand, is that such "NAIVE" C++ 
string code may not be performant, but in C++, even if you make 
allocations/deallocations during the game loop, this won't be good for 
the game performance, but that's all.


This is the reason why most game people shy away from D code. Because 
the GC is so inherent in the language, it's difficult to prove that you 
can call any function without incurring a possible GC cycle.


The two ways around this are to use @nogc, or to turn off the GC when 
you don't want it to run.


The blog has a whole series on working with and without the GC in D.



With D, ANY forgotten allocation during the game loop (and I really mean 
even JUST ONE hidden allocation somewhere in the whole game or engine), 
may cause the game to regularly freeze at the wrong time, because of an 
unwanted GC. Hence the phobia.


This is why you use @nogc. You then can't forget such things. But then 
of course, you need to avoid a lot of D niceties.


-Steve


Re: I have a plan.. I really DO

2018-07-06 Thread Steven Schveighoffer via Digitalmars-d-announce

On 7/6/18 11:19 AM, Ecstatic Coder wrote:

On Friday, 6 July 2018 at 14:52:46 UTC, 12345swordy wrote:

On Friday, 6 July 2018 at 14:11:05 UTC, Ecstatic Coder wrote:


Just one silly question.

Can the following "naive" D code trigger a garbage collection stall ?

score.Text = point_count.to!string() ~ " POINTS";

The correct answer is: I don't know, as I don't know what 
"point_count" is in the first place, as it never been defined.




Actually you answer was right even if the point count was not stored as 
an integer ;)


The real answer is what rikki says.

Note that what point_count is is irrelevant, the concatenation operator 
is going to trigger a GC allocation.


We could go further and say "Well, you haven't defined `to` or `string`" 
and then we can say we have no idea what this means at all!




For C++, the answer is : never.


Of course, the answer in C++ is that it won't compile, this is D code! ;)

And yes, you can have @nogc code in D, and it's less pleasant than 
normal D code, but still WAY more pleasant that most C++ code. So will 
people put up with this who want to just write games? Given the current 
state of the landscape, probably not.


At some point, someone is going to make a fantastic game engine in D, 
and then we will have a ballgame. Maybe it happens after D gets better 
support for reference counting, maybe before. Nothing inherent in D 
makes it impossible or unlikely. But people aren't going to switch "just 
because", there needs to be a compelling reason that causes someone to 
champion it.


-Steve


Re: DVM - D Version Manager 0.4.4

2018-07-02 Thread Steven Schveighoffer via Digitalmars-d-announce

On 7/2/18 3:36 AM, Tony wrote:

On Monday, 2 July 2018 at 07:12:47 UTC, Basile B. wrote:

On Monday, 2 July 2018 at 06:21:53 UTC, Tony wrote:

On Sunday, 13 September 2015 at 16:26:04 UTC, Jacob Carlborg wrote:

I just released a new version of DVM, 0.4.4. The most important


I am on Windows 10. Is:

dvm --latest install

a valid way to get the latest dmd? When I try that I get an exception




Seems unmaintained. Try Cybershadow's Digger which can handle building 
several versions of DMD too (https://github.com/CyberShadow/Digger), 
even from locally served webpage as UI.


OK, thanks! I saw DVM mentioned in a thread recently and I went back and 
couldn't find it and found this one via a search. The one I saw may have 
been a very old thread that someone revived.


I still use dvm (this version 0.4.4 is the latest I believe).

It still works, at least on OSX. But the errors it throws are not very 
user friendly, most of the time you get a stack trace.


I've never used the --latest switch (which BTW fails on OSX as well). 
Likely it's a change in how the metadata is stored on the server. Just 
install by name:


dvm install 2.080.1

-Steve


Boston D presentation Wednesday 6/27

2018-06-25 Thread Steven Schveighoffer via Digitalmars-d-announce

Hi all,

We are going to meet on 6/27 at 6 pm at the Capital One cafe in the Back 
Bay in Boston (https://www.capitalone.com/local/boston-backbay). Sameer 
Pradhan, a regular at the Boston D meetup group will show us his NLP 
tool that he is planning to port from Python to D, called OntoNotes.


I'm sure we will also find some place for drinks/dinner afterward, and 
I'd love to see some new D-velopers in our area, come and join us!


Note: I'm planning to make the Boston D meetings have a more regular 
schedule: at least once a month, with or without any presentations. I 
want to make sure the group stays together, and I think we have some 
good resources in our area to talk about how D can be used to help 
anyone with any project. Kind of like an AMA in person :)


On that note, if anyone has any ideas for good places to hang out and 
talk D, I'm open to suggestions. The cafe is nice, but I'm also think it 
would be nice to have places like the hotels we hang out and code at dconfs.


-Steve


Re: iopipe v0.1.0 - now with Windows support!

2018-06-19 Thread Steven Schveighoffer via Digitalmars-d-announce

On 6/19/18 7:18 AM, Jacob Carlborg wrote:

On 2018-06-11 16:45, Steven Schveighoffer wrote:


I just pushed v0.1.1 -- I realized that I never *actually* compiled on
windows, and there were a couple things that didn't work.

Note: the examples still don't work as they rely on openDev, which is
only available on Posix systems now.

I need to figure out a good way to open stdin/stdout in a cross platform
way with std.io.


You should setup AppVeyor [1] to make it works on Windows (when it works).

[1] https://www.appveyor.com



I just set up travis to do the Linux/mac testing. I need to add appveyor 
as well, but haven't gotten to it. I'm a complete CI noob, so I'm 
learning slowly :)


-Steve


Re: Encouraging preliminary results implementing memcpy in D

2018-06-14 Thread Steven Schveighoffer via Digitalmars-d-announce

On 6/14/18 9:31 AM, rikki cattermole wrote:

On 15/06/2018 1:22 AM, Steven Schveighoffer wrote:
It's really the impersonation that is the problem, not the anonymity. 
If you just would stick to one persona, and especially not impersonate 
people who actually post on this forum, you would not have to use Tor 
at all, and the forum moderators wouldn't have to worry about active 
moderation.


Not quite. It was the attacking of the D community at large which 
convinced those with the power to actively ban, to ban "him".


Well, for me, it wasn't attacks. There have been quite a few people in 
this community who were rude, insulting, and IMO, that is not worth 
moderation. In fact some of them have even came around and become quite 
good D contributors. Those problems usually work themselves out because 
we have a great community which does not give trolls the attention they 
desire.


But when you start impersonating people, especially ones that post to 
this forum, you have crossed the line and are literally putting words 
into other's mouths. We've seen this before, and generally they go away, 
but this guy does not want to.


Now he's claiming "discrimination" ;)

Once they are gone, we can deactivate the extra protections that have 
been put in place, because until this person came along, we were quite 
happy moderating ourselves. It lasted nearly 20 years that peace...


I agree, this will be a blip in our forum experience, and next month 
we'll barely remember him.


-Steve


Re: Encouraging preliminary results implementing memcpy in D

2018-06-14 Thread Steven Schveighoffer via Digitalmars-d-announce

On 6/13/18 10:32 PM, errExit wrote:

On Wednesday, 13 June 2018 at 17:04:11 UTC, Ali Çehreli wrote:


I am part of the D community. I haven't discriminated against anyone. 
I don't know what a Tor user is.


I've just searched: So Tor is an old idea of mine, implemented. :o)

Ali


Tor is our last line of defence against an Orson Wells future, where 
everyones actions are scrutinized by big brother, so that big brother 
can use that knowledge to put fear into, control and manipulate, those 
that don't conform.


Sorry KingJoffrey/psychoticRabbit/etc, you literally only started using 
Tor when your steady IP was banned. This argument kind of falls down 
when your past behavior is examined.


It's really the impersonation that is the problem, not the anonymity. If 
you just would stick to one persona, and especially not impersonate 
people who actually post on this forum, you would not have to use Tor at 
all, and the forum moderators wouldn't have to worry about active 
moderation.


-Steve


Re: Encouraging preliminary results implementing memcpy in D

2018-06-13 Thread Steven Schveighoffer via Digitalmars-d-announce

On 6/13/18 2:46 AM, Mike Franklin wrote:
I had a little fun today kicking the crap out of C's memcpy with a D 
implementation.


https://github.com/JinShil/memcpyD

Request for help: I don't have a Linux system running on real hardware 
at this time, nor do I have a wide range of platforms and machines to 
test with.  If you'd like to help me with this potentially foolish 
endeavor, please run the program on your hardware and send me the results.


Just FYI, Linux results on a VM are still valid -- many people 
(including myself) only use Linux on a VM, so even though it's not a 
definitive win against glibc memcpy on Linux, the end result is, it's 
still faster for those users :)


I won't add much, since I'm using a Mac, and those numbers have already 
been posted.


But I wanted to say don't discount those findings off-hand.

-Steve


Re: iopipe v0.1.0 - now with Windows support!

2018-06-13 Thread Steven Schveighoffer via Digitalmars-d-announce

On 6/13/18 12:03 PM, Steven Schveighoffer wrote:

I'm going to push this (I'll do some tests for the other widths to make 
sure it works for all UTF), but if you have any more things you want to 
work at CTFE, submit some issues on the github project.


v0.1.2 released

-Steve


Re: iopipe v0.1.0 - now with Windows support!

2018-06-13 Thread Steven Schveighoffer via Digitalmars-d-announce

On 6/13/18 8:35 AM, bauss wrote:

Does iopipe work with CTFE?


It may work in some cases. Some of the things it does are not conducive 
to CTFE working well -- like using a buffer. But generally at compile 
time, you don't want to use a buffer.


But I would expect, for instance, using algorithms and iopipes on a 
string would probably work.


hm... just thought I'd try it:

import std.range : walkLength;
static 
assert("hello\nworld\nthis\nis\na\ntest".byLineRange.walkLength == 6);


It didn't work at first, as I'm using memchr for searching for the 
newlines, but with a nice little if(__ctfe) override, it works just great!


I'm going to push this (I'll do some tests for the other widths to make 
sure it works for all UTF), but if you have any more things you want to 
work at CTFE, submit some issues on the github project.


I don't expect a lot of the array casting stuff to work, but maybe there 
are ways. I actually never thought much about making it work at 
compile-time. I suppose though, it would be cool to import a file at 
compile time, and generate the JSON or XML DOM at compile time too :)


I think zip and unzip are never going to work since the underlying C 
calls are not CTFE-able.


-Steve


Re: Seoul D Meetup

2018-06-12 Thread Steven Schveighoffer via Digitalmars-d-announce

On 6/11/18 10:50 PM, Mike Parker wrote:

Woohoo! I'm extremely pleased to announce the first Seoul D Meetup!

The three known D enthusiasts currently in Seoul (me, Mike Franklin, and 
Mathias Lang), and at least one potential enthusiast, are getting 
together at Charlie's (the hot dog shop my wife and I started a few 
years ago) to talk about our favorite programming language and future 
Seoul meetups.


If there's anyone else out there in Korea interested in joining us, 
we're meeting on June 20th at 7:00 pm. Free hot dogs and at least a 
couple of rounds of beer on me.


https://www.google.com/maps/@37.5336799,127.001524,3a,75y,215.2h,88.79t/data=!3m6!1e1!3m4!1sDJJPLRAaW6XHi5OM8XA90g!2e0!7i13312!8i6656 



This is awesome! I too wish I could be there. Dconf happens way too 
infrequently...


-Steve


Re: iopipe v0.1.0 - now with Windows support!

2018-06-12 Thread Steven Schveighoffer via Digitalmars-d-announce

On 6/12/18 10:19 AM, Jesse Phillips wrote:
I plan to eventually finish the JSON parser for a releasable state, 
and eventually tackle XML and a few other things.




You should definitely tackle xml by branching dxml. I'm really liking 
the api.


Of course that is my plan! I would never want to have to build an xml 
parser from scratch :)


-Steve


Re: iopipe v0.1.0 - now with Windows support!

2018-06-12 Thread Steven Schveighoffer via Digitalmars-d-announce

On 6/12/18 3:08 AM, Anton Fediushin wrote:

On Sunday, 10 June 2018 at 20:10:31 UTC, Steven Schveighoffer wrote:

iopipe version 0.1.0 has been released.

iopipe is a high-performance pipe processing system that makes it easy 
to string together pipelines to process data with as little buffer 
copying as possible.




I saw iopipe a while back, but never looked at it closely. Now I did 
and... it implements its own kind of pattern and not ranges?


Correct, although it's very similar to ranges.

I guess it 
is done for the sake of performance, but how easy it is to use iopipe 
with standard range-based methods from std.algorithm for example?


Very simple. You just have to define what is an "element" of a range 
that is a sliding window of data.


For example: 
https://github.com/schveiguy/iopipe/blob/master/source/iopipe/textpipe.d#L542


Note that asInputRange simply treats the current window as "front", and 
"popFront" discards that window and loads the next. It's a crude but 
effective tool to convert any iopipe into a range.


The reason iopipe is not based on ranges exactly (the window must be a 
random-access range), is because ranges don't handle i/o very 
performantly. E.g. I would never use lockingTextReader to process text 
data, it would be slow as hell, and too limiting.


File.byLine is fast, but only because of the underlying non-range tricks 
it uses to achieve performance. And iopipe still is 2x faster.


As long as it's easy to use with the rest of the phobos - I'd like to 
see it in the standard library.


It should be easy to use on its own, and with algorithms from phobos. 
I've done so in some of the toy parsers I've written.


I think the one sticking point (which I'm not sure how to reconcile, but 
we can probably hash it out) is that for iopipes, char and wchar arrays 
are arrays, not auto-decoding ranges.


-Steve


Re: iopipe v0.1.0 - now with Windows support!

2018-06-12 Thread Steven Schveighoffer via Digitalmars-d-announce

On 6/12/18 1:51 AM, DigitalDesigns wrote:


Could you explain some benefits specific to this implementation and a 
bit of the functional aspects for a proper overview of it's capabilities 
and why I should chose this method over others?


The things that I think make this approach better are:

1. Direct buffer access

Direct buffer access I have found is one of those ideas that doesn't 
seem like it's that impressive until you start using it. Many times, 
buffering is used in a generic library for optimization (basically 
amortizing reads) and is an implementation detail hidden from your view. 
Think of how FILE * keeps a large buffer of data inside itself, but only 
gives you access to one character at a time.


This forces you to create your *own* buffering scheme on top of that. 
What a waste! Iopipe allows you to use buffering for your purposes on 
top of the benefits of amortization. It's my belief that this is why 
iopipe's byline feature is 2x faster than Phobos'.


2. Using templates to their fullest

Iopipes are all templated on the buffer or iopipe underneath it. This 
makes tings easily swappable. It's really cool to be able to take your 
JSON or XML parser, and hook it onto an in-memory string in one line, 
and then hook it onto a socket, and everything is optimized for that 
situation. It takes the fun and flexibility of range programming and 
brings it to i/o.


This is why iopipe's byline handles all forms of UTF, compared to Phobos 
which only handles UTF8.


For example, I handle all forms of UTF with iopipe, with a decent set of 
utilities. Here is a complete program using iopipe that converts any 
type of UTF into another type, optimized for the specific situation:


https://github.com/schveiguy/iopipe/blob/master/examples/convert/convert.d

3. Compiler optimization for everything

All parts of iopipe, except for the low-level reads and writes (which 
ironically are not really part of iopipe) are visible to the compiler 
for inlining and optimization. I'm leveraging the power of the decades 
of optimization experience that the compiler can provide. This makes it 
easy to write code that performs well.


An anecdote: For my talk on iopipe in 2017 
(http://dconf.org/2017/talks/schveighoffer.html) I wanted to have a live 
demo showing the performance power. I literally was still working on it 
2 or 3 days before, while at dconf. I had already written a JSON parser, 
which was part of my presentation, but when I was showing it to another 
D user (Daniel Murphy), I couldn't really answer the question "how does 
it perform?". So he gave me a challenge -- do pretty printing on a JSON 
file. Simple enough, with the parser I had already written, took me 
about 1 hour to write it. It performed poorly compared to what we would 
have expected, but tweaking a few things (almost all were due to using 
some algorithms incorrectly), I got it to go faster than RapidJson in 
certain use cases, and reasonably close in others. And I did nothing in 
terms of lookup tables or using any special instructions. All in all, it 
was probably 2 hours of work, and the code is beautiful IMO! 
https://github.com/schveiguy/jsoniopipe/blob/master/examples/formatjson/formatjson.d


I think anyone who is doing parsing should have a look at iopipe, it not 
only may make your code much simpler and easier to read and write, but 
it would help me tune iopipe to cater to parsers, which I think is its 
wheelhouse.


I plan to eventually finish the JSON parser for a releasable state, and 
eventually tackle XML and a few other things.


-Steve


Re: DasBetterC: Converting make.c to D

2018-06-11 Thread Steven Schveighoffer via Digitalmars-d-announce

On 6/11/18 10:21 AM, Mike Parker wrote:
Walter's latest post on -betterC is now on the blog. Here, he shows 
step-by-step an example of using -betterC to convert a real-world 
program, one small enough to describe in a blog post, from C to D.


The blog:
https://dlang.org/blog/2018/06/11/dasbetterc-converting-make-c-to-d/

Reddit:
https://www.reddit.com/r/programming/comments/8q9u5t/dasbetterc_converting_makec_to_d/ 



"static if can replace cmany uses of #if"

cmany => many

-Steve


Re: iopipe v0.1.0 - now with Windows support!

2018-06-11 Thread Steven Schveighoffer via Digitalmars-d-announce

On 6/10/18 4:10 PM, Steven Schveighoffer wrote:

Nothing has really been changed, but it now has Windows i/o support. I 
will note at this time, however, that ring buffers are not yet supported 
on Windows.




I just pushed v0.1.1 -- I realized that I never *actually* compiled on 
windows, and there were a couple things that didn't work.


Note: the examples still don't work as they rely on openDev, which is 
only available on Posix systems now.


I need to figure out a good way to open stdin/stdout in a cross platform 
way with std.io.


-Steve


Re: iopipe v0.1.0 - now with Windows support!

2018-06-11 Thread Steven Schveighoffer via Digitalmars-d-announce

On 6/11/18 6:14 AM, Dejan Lekic wrote:

On Sunday, 10 June 2018 at 20:10:31 UTC, Steven Schveighoffer wrote:

iopipe version 0.1.0 has been released.

iopipe is a high-performance pipe processing system that makes it easy 
to string together pipelines to process data with as little buffer 
copying as possible.


All I can say (again, like I repeated on IRC many times) is that iopipe 
should be in the current form, or another, in Phobos. I just love it!


Thanks for the kind words! If Martin's std.io makes it in to Phobos, 
there is a realistic chance iopipe could go as well.


Just curious, do you have any projects based on iopipe?

-Steve


iopipe v0.1.0 - now with Windows support!

2018-06-10 Thread Steven Schveighoffer via Digitalmars-d-announce

iopipe version 0.1.0 has been released.

iopipe is a high-performance pipe processing system that makes it easy 
to string together pipelines to process data with as little buffer 
copying as possible.


Nothing has really been changed, but it now has Windows i/o support. I 
will note at this time, however, that ring buffers are not yet supported 
on Windows.


This version deprecates the IODev type that I had included, in favor of 
a new io library that shows extreme promise.


This version ONLY builds on 2.080.1 or later (the bug fix that I 
submitted at dconf has been merged in that version, and so iopipe will 
now build against Martin Nowak's io library). In fact, iopipe 
development was kind of stalled due to this bug, so I'm super-happy to 
see it fixed and released!


Note that the new io library also supports sockets, which IODev did not 
have support for, AND has a pluggable driver system, so you could 
potentially use fiber-based async io without rebuilding. It just makes a 
lot of sense for D to have a standard low-level io library that 
everything can use without having to kludge together multiple types of 
io libraries.


Near future plans:

1. Utilize a CI to make sure it continues to work on all platforms.
2. Add RingBuffer support on Windows
3. Add more driver support for std.io.
4. Continue development of JSON library that depends on iopipe (not yet 
on code.dlang.org).


git - https://github.com/schveiguy/iopipe
dub - https://code.dlang.org/packages/iopipe
docs - http://schveiguy.github.io/iopipe/

-Steve


Re: Beta 2.080.1

2018-06-08 Thread Steven Schveighoffer via Digitalmars-d-announce

On 6/8/18 2:13 PM, Russel Winder wrote:

On Mon, 2018-06-04 at 17:14 +0100, Russel Winder wrote:

On Mon, 2018-06-04 at 11:14 -0400, Steven Schveighoffer via
[…]


I just submitted a PR to fix
https://issues.dlang.org/show_bug.cgi?id=18934

I used stable. I'm hoping it could get in for this release.

https://github.com/dlang/phobos/pull/6544



So am I.



Looks like this didn't get into 2.080.1. When is 2.080.2 being released?


When or if is up to Martin.

If not 2.080.2, I will target master for 2.081. Seems like it was pretty 
much good to go, but didn't make the cutoff.


-Steve



Re: Driving Continuous Improvement in D

2018-06-05 Thread Steven Schveighoffer via Digitalmars-d-announce

On 6/5/18 9:58 AM, Steven Schveighoffer wrote:

On 6/5/18 3:20 AM, drug wrote:

04.06.2018 21:08, Steven Schveighoffer пишет:

On 6/4/18 1:51 PM, Joakim wrote:

On Monday, 4 June 2018 at 15:52:24 UTC, Steven Schveighoffer wrote:

On 6/2/18 3:23 AM, Mike Parker wrote:

[...]


I like the article, but was taken aback a bit by this quote: "for 
example, a PR to fix a bug in a specific piece of code mustn’t also 
edit the documentation of that function."


[...]


I think he was talking about _unrelated_ doc changes.


Well, how unrelated? If, for instance, you are changing the docs to 
accommodate the new code, and notice a typo, I would be fine with 
fixing that, and have even ASKED for that. I guess I need a bigger 
clarification, as the way it reads is that we require people split 
their doc changes from their code changes, and that simply hasn't 
been the case.




But what if your commit with this typo would be reverted? Then you 
lost your typo fix too.


Then you fix the typo again? Reverts don't happen enough to justify this 
concern.


To clarify a bit, complicated or controversial changes that are likely 
to be delayed or stalled, should be split from simple doc changes if it 
turns out it's not going to be pulled anytime soon. But normally, adding 
fixes for docs I would think is fine.


-Steve


Re: Driving Continuous Improvement in D

2018-06-05 Thread Steven Schveighoffer via Digitalmars-d-announce

On 6/5/18 3:20 AM, drug wrote:

04.06.2018 21:08, Steven Schveighoffer пишет:

On 6/4/18 1:51 PM, Joakim wrote:

On Monday, 4 June 2018 at 15:52:24 UTC, Steven Schveighoffer wrote:

On 6/2/18 3:23 AM, Mike Parker wrote:

[...]


I like the article, but was taken aback a bit by this quote: "for 
example, a PR to fix a bug in a specific piece of code mustn’t also 
edit the documentation of that function."


[...]


I think he was talking about _unrelated_ doc changes.


Well, how unrelated? If, for instance, you are changing the docs to 
accommodate the new code, and notice a typo, I would be fine with 
fixing that, and have even ASKED for that. I guess I need a bigger 
clarification, as the way it reads is that we require people split 
their doc changes from their code changes, and that simply hasn't been 
the case.




But what if your commit with this typo would be reverted? Then you lost 
your typo fix too.


Then you fix the typo again? Reverts don't happen enough to justify this 
concern.


-Steve


Re: Hunt framework 1.0.0 released

2018-06-05 Thread Steven Schveighoffer via Digitalmars-d-announce

On 6/5/18 3:25 AM, Brian wrote:


source code in github https://github.com/huntlabs/hunt/
documents in wiki https://github.com/huntlabs/hunt/wiki/
hunt framework website http://www.huntframework.com/


Is there a way to view your website in English? I found a popup on the 
bottom that has "English" as a selection, but it doesn't do anything.


-Steve


Re: GitHub could be acquired by Microsoft

2018-06-04 Thread Steven Schveighoffer via Digitalmars-d-announce

On 6/4/18 2:46 PM, Anton Fediushin wrote:

Of course MS does, since they spent $5 billion on it. They will try 
their best to make profit out of it, just like they did with LinkedIn.


$7.5 billion.

-Steve


Re: GitHub could be acquired by Microsoft

2018-06-04 Thread Steven Schveighoffer via Digitalmars-d-announce

On 6/3/18 11:51 PM, Anton Fediushin wrote:
This is still just a rumour, we'll know the truth on Monday (which is 
today).


Some articles about the topic:

https://fossbytes.com/microsoft-github-aquisition-report/
https://www.theverge.com/2018/6/3/17422752/microsoft-github-acquisition-rumors 



What's your opinion about that? Will you continue using GitHub?


Of course.

Both GitLab and Bitbucket can be used instead to host your D projects - 
dub registry supported them for a while now.


I use both bitbucket and github. I think I will simply continue to use 
what makes sense at the time (as Jonathan pointed out, hosting a private 
repository is free on bitbucket).


IMHO Microsoft isn't the type of company I want to see behind the 
GitHub. Maybe I am wrong since Microsoft has both money and programmers 
to improve it further, I just don't trust them too much which is the 
right thing to do when dealing with companies. This means that I will 
move my repositories elsewhere and use GitHub just to contribute to 
other projects.


I don't know if it makes any difference to me. Sure, they have 
infrastructure and market share, but all that changes if they do 
something really annoying. There are good competing sites, and people 
will just move their stuff. I'm sure it wouldn't take long for someone 
to make software that ports your entire github project to gitlab or 
whatever, maybe it already exists.


Microsoft just isn't the same big bad company that once paid for Linux 
licenses from SCO group to fund their lawsuit against Linux. This past 
year, they actually incorporated part of Linux into their OS! I don't 
think this is necessarily going to be bad for github.


One thing I have read that is intriguing: if you are a Microsoft 
competitor and you have private-source repos at github, how do you feel 
about that?


-Steve


Re: Driving Continuous Improvement in D

2018-06-04 Thread Steven Schveighoffer via Digitalmars-d-announce

On 6/4/18 1:51 PM, Joakim wrote:

On Monday, 4 June 2018 at 15:52:24 UTC, Steven Schveighoffer wrote:

On 6/2/18 3:23 AM, Mike Parker wrote:

[...]


I like the article, but was taken aback a bit by this quote: "for 
example, a PR to fix a bug in a specific piece of code mustn’t also 
edit the documentation of that function."


[...]


I think he was talking about _unrelated_ doc changes.


Well, how unrelated? If, for instance, you are changing the docs to 
accommodate the new code, and notice a typo, I would be fine with fixing 
that, and have even ASKED for that. I guess I need a bigger 
clarification, as the way it reads is that we require people split their 
doc changes from their code changes, and that simply hasn't been the case.


-Steve


Re: Driving Continuous Improvement in D

2018-06-04 Thread Steven Schveighoffer via Digitalmars-d-announce

On 6/2/18 3:23 AM, Mike Parker wrote:
In this post for the D Blog, Jack Stouffer details how dscanner is used 
in the Phobos development process to help improve code quality and fight 
entropy.


The blog:
https://dlang.org/blog/2018/06/02/driving-continuous-improvement-in-d/

reddit:
https://www.reddit.com/r/programming/comments/8nyzmk/driving_continuous_improvement_in_d/ 



I like the article, but was taken aback a bit by this quote: "for 
example, a PR to fix a bug in a specific piece of code mustn’t also edit 
the documentation of that function."


Really? I both was not aware of this policy, and don't understand why 
you wouldn't fix the docs at the same time. Can you elaborate?


I'll give you an example of what I was thinking of. Let's say you have a 
function foo:


/**
 * foo takes a parameter and returns true if ...
 */
bool foo(T)(T t) { ... }

And you realize foo really should only take integer parameters:

/**
 * foo takes integer parameters and returns true if ...
 */
bool foo(T)(T t) if (isIntegral!T) { ... }

Why not both edit the function and fix the docs in the same PR? In fact, 
why would we *accept* the change without updating the docs?


-Steve


Re: Beta 2.080.1

2018-06-04 Thread Steven Schveighoffer via Digitalmars-d-announce

On 6/4/18 7:44 AM, Martin Nowak wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

First beta for the 2.080.1 patch release.

Comes with a handful of fixes.

http://dlang.org/download.html#dmd_beta
http://dlang.org/changelog/2.080.1.html

Please report any bugs at https://issues.dlang.org

- -Martin
-BEGIN PGP SIGNATURE-

iQIzBAEBCgAdFiEEpzRNrTw0HqEtE8TmsnOBFhK7GTkFAlsVJfgACgkQsnOBFhK7
GTkr5Q//Se1gg7PfPoywWLdBkiO7sSqaZf0BFsCXFbaB8tnXaS0IZ7xyHej2Lun8
VNnWi1kPBXKBt6JLpcBU63u1Gl6IuBDVjPG4wJArb6QrzL73bL2DgBfASm+ZnzWb
VzxUJTmlaCIw6FsXAMEMy4L7p+VDHRN4zWBLgh24r2aEuJ5jacUyWRZG3L5MStAT
3QkOCjYpVsGg6QPklfJHbRI0zFv8qZKGP+ab1vOegfObkrcqF1g5y2e+Bigla8MJ
llhvJWBtmDJYklhDXZN3oOL66c2JykScHa9qArKb45Xk6wtSdCJGDhbpY2wrLHcc
4SnOd6ZxPQ0M0ON/bL5Fm7BNebT2QQZWTdasHVXMsokV9uW9FKX/BA1VOsuJvvb9
/mmaY1MUAb1S4y1+RoY6nfN9G8RH9S5vujV9F4kWjnH0J3rc66lPMfOnOU1Wckup
APaXg2L10sZaQ4Z+a4Gh5a5synWkUJB09q4jBKb90gTUsiN6Erp4GCAZ6PKRbkJ1
Z6jK8yAgrstCi7ctg0QYrH6EGBqigAP13itbBKfOmB0DH010oNY1i9GB9vc1zvSM
jpdoCYX1pe4ljeMZXZDiTyGTM5g4TklBsdvwC5PTuvvFKNvU4K8RKg6Zy4FqHq0R
bEl63PWSmE3hrMBjrk41qxw/NEs5UgtkPYBl1aWv49+EfIhXMyA=
=W/Qn
-END PGP SIGNATURE-



I just submitted a PR to fix
https://issues.dlang.org/show_bug.cgi?id=18934

I used stable. I'm hoping it could get in for this release.

https://github.com/dlang/phobos/pull/6544

-Steve


Re: DConf 2018 Ex Post Facto

2018-06-01 Thread Steven Schveighoffer via Digitalmars-d-announce

On 5/31/18 7:06 PM, Stefan wrote:

On Thursday, 31 May 2018 at 19:31:20 UTC, Steven Schveighoffer wrote:


There was a small "presentation" where people could show what they had 
accomplished during the hackathon, which gives a rough estimation of 
what you are looking for. It was a google doc, but I have no idea 
where it is now.


Hackathon google doc:
https://docs.google.com/document/d/1qrn7JZS62hzsylpGM1VhaT6YW2I1NVDDGQsANv5b1GQ/edit?usp=drivesdk 





Yep that's it, the first part was where we added our names to show what 
we did. I guess reading it now, it looks a lot less informative than I 
thought :)


-Steve


Re: DConf 2018 Ex Post Facto

2018-05-31 Thread Steven Schveighoffer via Digitalmars-d-announce

On 5/31/18 1:40 PM, jmh530 wrote:

On Thursday, 31 May 2018 at 15:01:12 UTC, Mike Parker wrote:
Since I returned home from my extended trip to Germany, it's been a 
slog trying to ramp back up into my usual routine. It was a week 
before I could find any words at all for a retrospective on the 
conference, and it very nearly took another week to get the post in 
readable form. I'm still not at peak productivity, but I'm getting 
there. I've got a couple of guest posts lined up (including one from 
Walter) and I should be getting the Twitter & FB feeds going again soon.


In the meantime, here's what DConf 2018 was like from my perspective.

The blog:
https://dlang.org/blog/2018/05/31/dconf-2018-ex-post-facto/

Reddit:
https://www.reddit.com/r/d_language/comments/8nj1nn/dconf_2018_ex_post_facto/ 



Might be interesting (perhaps next year) to do a post-mortem for the 
Hackathon where each participant writes one or two sentences on what 
they accomplished. You can then aggregate together into a blog post.


There was a small "presentation" where people could show what they had 
accomplished during the hackathon, which gives a rough estimation of 
what you are looking for. It was a google doc, but I have no idea where 
it is now.


-Steve


Re: Complicated Types: Prefer “alias this” Over “alias” For Easier-To-Read Error Messages

2018-05-21 Thread Steven Schveighoffer via Digitalmars-d-announce

On 5/21/18 4:29 PM, Nick Sabalausky (Abscissa) wrote:

On 05/21/2018 01:30 PM, Paul Backus wrote:


I'm not sure making `_data` private is really a good idea. For 
example, this only works if `_data` is public:


import std.algorithm;
import std.range;
import std.stdio;

struct MyType
{
 auto _data = iota(10);
 alias _data this;
}

void main()
{
 MyType x;
 x.each!writeln;
}


Ouch, and the error message isn't very helpful either. I wonder what 
causes this to fail though, and whether it might simply be a compiler bug?


No, an alias to private data doesn't magically make it public.

The reason it doesn't work is that each is in std.algorithm.iteration, 
which has no visibility into private symbols of your main module.


This also fails:

assert(isInputRange!MyType);

-Steve


Re: Complicated Types: Prefer “alias this” Over “alias” For Easier-To-Read Error Messages

2018-05-21 Thread Steven Schveighoffer via Digitalmars-d-announce

On 5/21/18 10:48 AM, Mike Parker wrote:
Nick Sabaluasky's first post to the D Blog is a tip on how to create an 
aliased type that keeps its name in error messages.


The blog:
https://dlang.org/blog/2018/05/21/complicated-types-prefer-alias-this-over-alias-for-easier-to-read-error-messages/ 



Reddit:
https://www.reddit.com/r/programming/comments/8l1a8u/prefer_alias_this_over_alias_for_easiertoread/ 



The list has two "1." headers.

Nice idea, I wonder if the compiler couldn't do this automatically with 
alias, however. It already does this in some cases (e.g. string instead 
of immutable(char)[]).


This would help solve the problem that error messages aren't going to 
get better when you pass it into something that only accepts the aliased 
type. However, for the most part, these are non-template functions anyway.


-Steve


Re: vibe.d 0.7.33 maintenance release

2018-05-21 Thread Steven Schveighoffer via Digitalmars-d-announce

On 5/20/18 8:07 AM, Sönke Ludwig wrote:

Another big improvement is the diet-ng library, which uses less 
compile-time memory and is a lot more flexible than the old 
`vibe.textfilter.diet` module.


Isn't diet-ng used by 0.7.x branch now? It seems to be selected when I'm 
using 0.7.x


-Steve


Re: iopipe v0.0.4 - RingBuffers!

2018-05-14 Thread Steven Schveighoffer via Digitalmars-d-announce

On 5/14/18 6:02 AM, bioinfornatics wrote:

On Thursday, 10 May 2018 at 23:22:02 UTC, Steven Schveighoffer wrote:
OK, so at dconf I spoke with a few very smart guys about how I can use 
mmap to make a zero-copy buffer. And I implemented this on the plane 
ride home.


However, I am struggling to find a use case for this that showcases 
why you would want to use it. While it does work, and works 
beautifully, it doesn't show any measurable difference vs. the array 
allocated buffer that copies data when it needs to extend.


If anyone has any good use cases for it, I'm open to suggestions. 
Something that is going to potentially increase performance is an 
application that needs to keep the buffer mostly full when extending 
(i.e. something like 75% full or more).


The buffer is selected by using `rbufd` instead of just `bufd`. 
Everything should be a drop-in replacement except for that.


Note: I have ONLY tested on Macos, so if you find bugs in other OSes 
let me know. This is still a Posix-only library for now, but more on 
that later...


As a test for Ring buffers, I implemented a simple "grep-like" search 
program that doesn't use regex, but phobos' canFind to look for lines 
that match. It also prints some lines of context, configurable on the 
command line. The lines of context I thought would show better 
performance with the RingBuffer than the standard buffer since it has 
to keep a bunch of lines in the buffer. But alas, it's roughly the 
same, even with large number of lines for context (like 200).


However, this example *does* show the power of iopipe -- it handles 
all flavors of unicode with one template function, is quite 
straightforward (though I want to abstract the line tracking code, 
that stuff is really tricky to get right). Oh, and it's roughly 10x 
faster than grep, and a bunch faster than fgrep, at least on my 
machine ;) I'm tempted to add regex processing to see if it still 
beats grep.


Next up (when my bug fix for dmd is merged, see 
https://issues.dlang.org/show_bug.cgi?id=17968) I will be migrating 
iopipe to depend on https://github.com/MartinNowak/io, which should 
unlock Windows support (and I will add RingBuffer Windows support at 
that point).


Enjoy!

https://github.com/schveiguy/iopipe
https://code.dlang.org/packages/iopipe
http://schveiguy.github.io/iopipe/



Hi Steve,

It is an exciting works, that could help in bioinformatics area.
Indeed in bioinformatics we are I/O bounding and we process lot of big 
files the amount of data can be in gigabytes, terabytes and even some 
times in petabytes.


So processing efficiently these amount of data is critic. Some years ago 
I got a request 'How to parse fastq file format in D?' and monarch_dodra 
wrote a really fast parser (code: http://dpaste.dzfl.pl/37b893ed )


It could be interesting to show how iopipe is fast.


Yeah, I have been working on and off with Vang Le (biocyberman) on using 
iopipe to parse such formats. He gave a good presentation at dconf this 
year on using D in bioinformatics, and I think it is a great fit for D!


At dconf, I threw together a crude fasta parser (with the intention of 
having it be the base for parsing fastq as well) to demonstrate how 
iopipe can perform while parsing such things. I have no idea how fast or 
slow it is, as I just barely got it to work (pass unit tests I made up 
based on wikipedia entry for fasta), but IMO, the direct buffer access 
makes fast parsing much more pleasant than having to deal with your own 
buffering (using phobos makes parsing a bit difficult, however, I still 
see a need for some parsing tools for iopipe).


You can find that library here: https://github.com/schveiguy/fastaq

Not being in the field of bioinformatics, I can't really say that I am 
likely to continue development of it, but I'm certainly willing to help 
with iopipe for anyone who wants to use it in this field.


-Steve


Re: iopipe v0.0.4 - RingBuffers!

2018-05-12 Thread Steven Schveighoffer via Digitalmars-d-announce

On 5/12/18 3:38 PM, Patrick Schluter wrote:

On Thursday, 10 May 2018 at 23:22:02 UTC, Steven Schveighoffer wrote:
OK, so at dconf I spoke with a few very smart guys about how I can use 
mmap to make a zero-copy buffer. And I implemented this on the plane 
ride home.


[...]


They can be problematic with some CPU's and OS's. For modern CPU's there 
should be no problems but those with exotic caches and virtual memory 
configurations there can be some aliasing issues. Linus Torvalds talked 
a little about that case in this thread of realworldtech

https://www.realworldtech.com/forum/?threadid=174426=174731


Thanks for the tip. The nice thing about iopipe is that the buffer type 
is completely selectable, and nothing changes, except possibly some 
performance.


So on those arch's, I would expect people to select the normal 
AllocatedBuffer type.


-Steve


Re: iopipe v0.0.4 - RingBuffers!

2018-05-11 Thread Steven Schveighoffer via Digitalmars-d-announce

On 5/11/18 11:44 AM, Steven Schveighoffer wrote:

On 5/10/18 7:22 PM, Steven Schveighoffer wrote:

However, this example *does* show the power of iopipe -- it handles 
all flavors of unicode with one template function, is quite 
straightforward (though I want to abstract the line tracking code, 
that stuff is really tricky to get right). Oh, and it's roughly 10x 
faster than grep, and a bunch faster than fgrep, at least on my 
machine ;) I'm tempted to add regex processing to see if it still 
beats grep.


Shameful note: Macos grep is BSD grep, and is not NEARLY as fast as GNU 
grep, which has much better performance (and is 2x as fast as 
iopipe_search on my Linux VM, even when printing line numbers).


So at least there is something to strive for :)


More testing reveals that as I increase the context lines to print, 
iopipe performs better than GNU grep. A shocking thing is that at 9 
lines of context, grep goes up slightly, but all of a sudden at 10 lines 
of context, it doubles in the time taken (and is now slower than the 
iopipe_search).


Also noting: my Linux VM does not have ldc, so these are dmd numbers.

-Steve


Re: iopipe v0.0.4 - RingBuffers!

2018-05-11 Thread Steven Schveighoffer via Digitalmars-d-announce

On 5/10/18 7:22 PM, Steven Schveighoffer wrote:

However, this example *does* show the power of iopipe -- it handles all 
flavors of unicode with one template function, is quite straightforward 
(though I want to abstract the line tracking code, that stuff is really 
tricky to get right). Oh, and it's roughly 10x faster than grep, and a 
bunch faster than fgrep, at least on my machine ;) I'm tempted to add 
regex processing to see if it still beats grep.


Shameful note: Macos grep is BSD grep, and is not NEARLY as fast as GNU 
grep, which has much better performance (and is 2x as fast as 
iopipe_search on my Linux VM, even when printing line numbers).


So at least there is something to strive for :)

-Steve


Re: iopipe v0.0.4 - RingBuffers!

2018-05-11 Thread Steven Schveighoffer via Digitalmars-d-announce

On 5/11/18 5:55 AM, Kagamin wrote:

On Thursday, 10 May 2018 at 23:22:02 UTC, Steven Schveighoffer wrote:
However, I am struggling to find a use case for this that showcases 
why you would want to use it. While it does work, and works 
beautifully, it doesn't show any measurable difference vs. the array 
allocated buffer that copies data when it needs to extend.


Depends on OS and hardware. I would expect mmap implementation to be 
slower as it reads file in chunks of 4kb and relies on page faults.


As Dmitry hinted at, there actually is no file involved. I'm mapping 
just straight memory to 2 segments. In fact, in my test application, I'm 
using stdin as the input, which may not even involve a file.


It's just as fast as using memory, the only cool part is that you can 
write a buffer that wraps to the beginning as if it were a normal array.


What surprises me is that the copying for the normal buffer doesn't hurt 
performance that much. I suppose this should probably have been 
expected, as CPUs are really really good at processing consecutive 
memory, and the copying you end up having to do is generally small 
compared to the rest of your app.


-Steve


Re: iopipe v0.0.4 - RingBuffers!

2018-05-11 Thread Steven Schveighoffer via Digitalmars-d-announce

On 5/11/18 1:30 AM, Dmitry Olshansky wrote:

On Thursday, 10 May 2018 at 23:22:02 UTC, Steven Schveighoffer wrote:
OK, so at dconf I spoke with a few very smart guys about how I can use 
mmap to make a zero-copy buffer. And I implemented this on the plane 
ride home.


However, I am struggling to find a use case for this that showcases 
why you would want to use it. While it does work, and works 
beautifully, it doesn't show any measurable difference vs. the array 
allocated buffer that copies data when it needs to extend.


I’d start with something clinicaly synthetic.
Say your record size is exactly half of buffer + 1 byte. If you were to 
extend the size of buffer, it would amortize.


Hm.. this wouldn't work, because the idea is to keep some of the buffer 
full. What will happen here is that the buffer will extend to be able to 
accomodate the extra byte, and then you are back to having less of the 
buffer full at once. Iopipe is not afraid to increase the buffer :)




Basically:
16 Mb buffer fixed
vs
16 Mb mmap-ed ring

Where you read pieces in 8M+1 blocks.Yes, we are aiming to blow the CPU 
cache there. Otherwise CPU cache is so fast that ocasional copy is 
zilch, once we hit primary memory it’s not. Adjust sizes for your CPU.


This isn't how it will work. The system looks at the buffer and says 
"oh, I can just read 8MB - 1 byte," which gives you 2 bytes less than 
you need. Then you need the extra 2 bytes, so it will increase the 
buffer to hold at least 2 records.


I do get the point of having to go outside the cache. I'll look and see 
if maybe specifying a 1000 line context helps ;)


Update: nope, still pretty much the same.

The amount of work done per byte though has to be minimal to actually 
see anything.


Right, this is another part of the problem -- if copying is so rare 
compared to the other operations, then the difference is going to be 
lost in the noise.


What I have learned here is:

1. Ring buffers are really cool (I still love how it works) and perform 
as well as normal buffers

2. The use cases are much smaller than I thought
3. In most real-world applications, they are a wash, and not worth the 
OS tricks needed to use it.
4. iopipe makes testing with a different kind of buffer really easy, 
which was one of my original goals. So I'm glad that works!


I'm going to (obviously) leave them there, hoping that someone finds a 
good use case, but I can say that my extreme excitement at getting it to 
work was depressed quite a bit when I found it didn't really gain much 
in terms of performance for the use cases I have been doing.


in the buffer. But alas, it's roughly the same, even with large number 
of lines for context (like 200).


However, this example *does* show the power of iopipe -- it handles 
all flavors of unicode with one template function, is quite 
straightforward (though I want to abstract the line tracking code, 
that stuff is really tricky to get right). Oh, and it's roughly 10x 
faster than grep, and a bunch faster than fgrep, at least on my 
machine ;) I'm tempted to add regex processing to see if it still 
beats grep.


Should be mostly trivial in fact. I mean our first designs for IOpipe is 
where I wanted regex to work with it.


Basically - if we started a match, extend window until we get it or lose 
it. Then release up to the next point of potential start.


I'm thinking it's even simpler than that. All matches are dead on a line 
break (it's how grep normally works), so you simply have to parse the 
lines and run each one via regex. What I don't know is how much it costs 
regex to startup and run on an individual line.


One thing I could do to amortize is keep 2N lines in the buffer, and run 
the regex on a whole context's worth of lines, then dump them all.


I don't get why grep is so bad at this, since it is supposedly doing the 
matching without line boundaries. I was actually quite shocked when 
iopipe was that much faster -- even when I'm not asking grep to print 
out line numbers (so it doesn't actually ever really have to keep track 
of lines).


-Steve


iopipe v0.0.4 - RingBuffers!

2018-05-10 Thread Steven Schveighoffer via Digitalmars-d-announce
OK, so at dconf I spoke with a few very smart guys about how I can use 
mmap to make a zero-copy buffer. And I implemented this on the plane 
ride home.


However, I am struggling to find a use case for this that showcases why 
you would want to use it. While it does work, and works beautifully, it 
doesn't show any measurable difference vs. the array allocated buffer 
that copies data when it needs to extend.


If anyone has any good use cases for it, I'm open to suggestions. 
Something that is going to potentially increase performance is an 
application that needs to keep the buffer mostly full when extending 
(i.e. something like 75% full or more).


The buffer is selected by using `rbufd` instead of just `bufd`. 
Everything should be a drop-in replacement except for that.


Note: I have ONLY tested on Macos, so if you find bugs in other OSes let 
me know. This is still a Posix-only library for now, but more on that 
later...


As a test for Ring buffers, I implemented a simple "grep-like" search 
program that doesn't use regex, but phobos' canFind to look for lines 
that match. It also prints some lines of context, configurable on the 
command line. The lines of context I thought would show better 
performance with the RingBuffer than the standard buffer since it has to 
keep a bunch of lines in the buffer. But alas, it's roughly the same, 
even with large number of lines for context (like 200).


However, this example *does* show the power of iopipe -- it handles all 
flavors of unicode with one template function, is quite straightforward 
(though I want to abstract the line tracking code, that stuff is really 
tricky to get right). Oh, and it's roughly 10x faster than grep, and a 
bunch faster than fgrep, at least on my machine ;) I'm tempted to add 
regex processing to see if it still beats grep.


Next up (when my bug fix for dmd is merged, see 
https://issues.dlang.org/show_bug.cgi?id=17968) I will be migrating 
iopipe to depend on https://github.com/MartinNowak/io, which should 
unlock Windows support (and I will add RingBuffer Windows support at 
that point).


Enjoy!

https://github.com/schveiguy/iopipe
https://code.dlang.org/packages/iopipe
http://schveiguy.github.io/iopipe/

-Steve


Re: Dconf live stream for Shachar's talk

2018-05-05 Thread Steven Schveighoffer via Digitalmars-d-announce

On 5/5/18 9:56 AM, Steven Schveighoffer wrote:

Hi all,

We are live streaming Shachar's talk this morning at 
https://www.youtube.com/watch?v=xNWRgEHxOhc


Ali is uploading the slides to dconf.org so you can follow along.

Cheers!


I will note that we don't have the normal A/V crew from the other days, 
so this is being done via a laptop camera. So you will probably need to 
download the slides.


-Steve


Dconf live stream for Shachar's talk

2018-05-05 Thread Steven Schveighoffer via Digitalmars-d-announce

Hi all,

We are live streaming Shachar's talk this morning at 
https://www.youtube.com/watch?v=xNWRgEHxOhc


Ali is uploading the slides to dconf.org so you can follow along.

Cheers!

-Steve


Re: #include C headers in D code

2018-04-10 Thread Steven Schveighoffer via Digitalmars-d-announce

On 4/10/18 2:36 PM, Atila Neves wrote:



Haha, I remember. I do plan on dealing with emplace_back, but I have no 
idea how just yet and I was hoping nobody was going to call me on it 
until then. Busted! :P




I think we all agree you aren't going to instantiate C++ templates in D 
(and who would want to).


But since you are using a wrapper to call the compiler, and invoking 
clang anyway, generating a file to compile which calls the template to 
generate the object file for the method, and then using the symbol 
generated to bind to the method might be possible. Of course, you first 
have to compile your D code to see what templates to generate! So this 
is another extra step.


It could be as easy as generating a mock C++ template for vector, that 
then has a symbol that d++ recognizes and both replaces with a real call 
AND helps you figure out what C++ template to generate. -vcg-ast may 
help here.


Whatever this is going to be, it ain't going to be fast...

-Steve


Re: #include C headers in D code

2018-04-10 Thread Steven Schveighoffer via Digitalmars-d-announce

On 4/10/18 12:51 PM, Atila Neves wrote:

On Tuesday, 10 April 2018 at 13:53:34 UTC, Steven Schveighoffer wrote:


If you get to the point where you can #include , it will be 
doubly impressive!


Not *if*, *when*. ;)


I hope you are right, but I remain skeptical :)

If I may throw a curveball back at you, I recall a question at last 
year's dconf during the talk about interfacing with C++ STL about using 
emplace_back instead of push_back. And the questioner who happened to be 
sitting next to me said "meh, push_back isn't very useful, nobody uses 
push_back any more" :P


-Steve


Re: #include C headers in D code

2018-04-10 Thread Steven Schveighoffer via Digitalmars-d-announce

On 4/9/18 7:03 AM, Atila Neves wrote:
Here's my blog post about my project that allows directly #including C 
headers in D*


https://atilanevesoncode.wordpress.com/2018/04/09/include-c-headers-in-d-code/ 



The summary is that, modulo bugs, things like this work:

     #include 
     void main() { printf("Hello world\n".ptr); }

So far it's successfully compiled whilst #including pthread, libcurl, 
openssl and others. The blog and the github README have more 
information, and feel free to reply to this with questions.


dub: http://code.dlang.org/packages/dpp
reddit: 
https://www.reddit.com/r/programming/comments/8axj53/include_c_headers_in_d_code/ 


hacker news: It's in there somewhere, search around.

Atila

* Technically, "D + #include directives + C macros"


Awesome. Can't say I will use it, as I don't use C much, but I 
understand how difficult a task this is.


If you get to the point where you can #include , it will be 
doubly impressive!


-Steve


Re: std.variant Is Everything Cool About D

2018-04-04 Thread Steven Schveighoffer via Digitalmars-d-announce

On 4/3/18 11:29 PM, Meta wrote:
Also, with Nullable your 
data is guaranteed to not be boxed, whereas it's a possibility with 
Variant/Algebraic if the types you're working with are large enough.


Not with Algebraic.

-Steve


Re: Why think unit tests should be in their own source code hierarchy instead of side-by-side

2018-03-27 Thread Steven Schveighoffer via Digitalmars-d-announce

On 3/26/18 9:26 AM, Atila Neves wrote:

On Friday, 23 March 2018 at 14:54:57 UTC, Steven Schveighoffer wrote:

On 3/22/18 6:59 AM, Atila Neves wrote:

Blog post:

https://atilanevesoncode.wordpress.com/

Atila


It's simple. Unittests in imported modules should not be visible. They 
should be compiled as if -unittest was not passed.


Even Walter and Andrei are supportive: 
https://github.com/dlang/dmd/pull/6375#issuecomment-373487247




That would completely break __traits(getUnitTests).


I'm sure we could find a way to keep the features here, 99.99% of the 
time, you don't care about, nor want to parse or semantic, an imported 
module's unit tests. Only specialized unit test frameworks care about 
this feature.


It could be as simple as, if you use __traits(getUnitTests, modulename) 
anywhere in a module, then that module is imported the traditional way. 
Or we could create a specialized "import unittest" syntax for this purpose.


I think we can have the best of both worlds, with the common case being 
preferred.


-Steve


Re: Why think unit tests should be in their own source code hierarchy instead of side-by-side

2018-03-24 Thread Steven Schveighoffer via Digitalmars-d-announce

On 3/23/18 9:54 PM, Tony wrote:

I said my "original reply", meaning the one where I first mentioned 
Test-Driven Development. That was to something that Steven Schveighoffer 
said (although I did not reply directly to his message, but replied to 
his comment that was still in H.S. Teoh's message):


"I've worked on a project where the testing was separated from the code, 
and it was a liability IMO. Things would get missed and not tested 
properly."


He doesn't explicitly specify development or maintenance, but I assume 
it was development.


It was somewhat development and somewhat maintenance, but I was not 
using TDD. The code was already written, I was adding to it. I was also 
adding tests afterwards to make sure what I did worked properly.


In my case, I had no choice but to use separate files, as I was writing 
firmware for an embedded device, and the tests were running on the host 
PC (i.e. testing that when I communicated with the device, it was doing 
what I expected).


But even in my normal work where tests and the code run on the same 
system, my style of development just doesn't fit TDD, I often times 
don't know what the code or functionality is going to look like at the 
end when I'm done (I rewrote what eventually became iopipe 5 times 
because I wasn't sure of the best way to do it). I can see where if you 
use TDD, it would be OK to separate the tests from the program, but I 
still feel that the cost of having them separated from the code it's 
testing outweighs the benefits of less recompilations. I'm just not as 
picky as Atila when it comes to build time :)


-Steve


Re: Why think unit tests should be in their own source code hierarchy instead of side-by-side

2018-03-23 Thread Steven Schveighoffer via Digitalmars-d-announce

On 3/23/18 3:46 PM, Johan Engelen wrote:

On Thursday, 22 March 2018 at 15:18:40 UTC, Jacob Carlborg wrote:

On Thursday, 22 March 2018 at 11:00:31 UTC, Atila Neves wrote:


Direct link:

https://atilanevesoncode.wordpress.com/2018/03/22/keep-d-unittests-separated-from-production-code/ 



I completely agree. Although my reason is mostly because there will be 
too much code in a single file if the regular code and unit tests are 
mixed in the same file.


Fully agree with this "too much code in a single file" point. I am 
confident that part of the reason of Phobos unittesting being very 
incomplete, is that adding unittests further clutters the codebase.
Moving all unittests to the bottom of the file (pulling them out of 
classes too) would resolve this issue in part.


Note that a frequent complaint of std.datetime (at least when it was one 
module) is that the file was too big. While it does hold a lot of 
functionality, the majority of the file size is unittests. This means 
that it can be hard to surf the file for functionality.


But on the flip side, there aren't a lot of datetime bugs!

I personally believe that there should be unit tests for every function, 
and be right next to the function. I don't want to go on a search for 
such things, or have to rely on manual documentation to know what is 
testing what. It would be nice to have your editor hide the unit tests 
unless you want to work on them.


I've worked on a project where the testing was separated from the code, 
and it was a liability IMO. Things would get missed and not tested properly.


-Steve


Re: Why think unit tests should be in their own source code hierarchy instead of side-by-side

2018-03-23 Thread Steven Schveighoffer via Digitalmars-d-announce

On 3/22/18 6:59 AM, Atila Neves wrote:

Blog post:

https://atilanevesoncode.wordpress.com/

Atila


It's simple. Unittests in imported modules should not be visible. They 
should be compiled as if -unittest was not passed.


Even Walter and Andrei are supportive: 
https://github.com/dlang/dmd/pull/6375#issuecomment-373487247


-Steve


  1   2   3   4   >