Re: String Template Package

2020-10-26 Thread H. S. Teoh via Digitalmars-d-learn
On Mon, Oct 26, 2020 at 11:36:12PM +, Per Nordlöw via Digitalmars-d-learn 
wrote:
> I need a string template system for generation of parsers in D code to
> compiled in separate phase (not Pegged).
> 
> What packages is there for this?

Adela Vais is working on D support for GNU bison, a parser generator
that uses a separate compile phase.  I don't know how far she has gotten
and how usable the current support is, but it's something to keep an eye
out for.


T

-- 
Let's call it an accidental feature. -- Larry Wall


Re: Recommended way to use __FILE__/__LINE__ etc.

2020-10-22 Thread H. S. Teoh via Digitalmars-d-learn
On Thu, Oct 22, 2020 at 03:32:57PM +, Andrey Zherikov via 
Digitalmars-d-learn wrote:
> There are two ways how __FILE__, __LINE__ etc. can se used in function
> parameters: as regular parameters and as template parameters:
[...]
> What is recommended way?
> What are pros/cons of each case?

I don't know what's the "recommended" way, but generally, I prefer to
pass them as regular parameters. Otherwise you will get a lot of
template bloat: one instantiation per file/line combination.

However, sometimes there is not much choice: if your function is
variadic, then you pretty much have to use template parameters, 'cos
otherwise you won't be able to pick up the __FILE__/__LINE__ of the
caller with default arguments.


T

-- 
Knowledge is that area of ignorance that we arrange and classify. -- Ambrose 
Bierce


Re: Forward referencing functions in D

2020-10-16 Thread H. S. Teoh via Digitalmars-d-learn
On Fri, Oct 16, 2020 at 08:04:07PM +, Imperatorn via Digitalmars-d-learn 
wrote:
[...]
> I think it might be just because you havent defined the function yet
> at that point.

That's not correct; the following works:

module mymodule;

void func() {
forwardfunc();
}

void forwardfunc() {
}

However, this only applies in module scope. If they were declared inside
a function body, forwardfunc must be declared before func.


T

-- 
It is impossible to make anything foolproof because fools are so ingenious. -- 
Sammy


Re: Forward referencing functions in D

2020-10-16 Thread H. S. Teoh via Digitalmars-d-learn
On Fri, Oct 16, 2020 at 07:55:53PM +, wilcro via Digitalmars-d-learn wrote:
> The web page "Programming in D for C Programmers"
> (https://dlang.org/articles/ctod.html#forwardfunc) states that forward
> declarations are neither required nor permitted,
[...]
> However, the following code will cause a compiler error:
[...]
> void main()
> {
> 
> void myfunc() {
> 
> forwardfunc(); // onlineapp.d(8): Error: undefined identifier
> forwardfunc
> }
> 
> void forwardfunc() {
> 
> writeln("foo");
> }
[...]

This is because order-independence of declarations only applies to
module scope, not to function scope.  So the above would work if myfunc
and forwardfunc were moved outside of main().  But inside a function
body, you must declare everything before you use them.


T

-- 
Philosophy: how to make a career out of daydreaming.


Re: why do i need an extern(C): here?

2020-10-15 Thread H. S. Teoh via Digitalmars-d-learn
On Thu, Oct 15, 2020 at 09:47:16PM +, IGotD- via Digitalmars-d-learn wrote:
> On Thursday, 15 October 2020 at 21:29:59 UTC, WhatMeWorry wrote:
> > 
> > I've go a small DLL and a test module both written in D. Why do I
> > need to use the extern(C)?  Shouldn't both sides be using D name
> > wrangling?
> > 
> 
> You have answered your own question. If you're not using extern(C), D
> just like C++ adds a lot of name mangling which is not "addSeven" but
> extra characters as well. GetProcAddress which is a Windows call has
> no idea about D name mangling and therefore will not find the
> function.

You can use .mangleof to get the mangled name of a D function that you
can then lookup with GetProcAddress.


T

-- 
Notwithstanding the eloquent discontent that you have just respectfully 
expressed at length against my verbal capabilities, I am afraid that I must 
unfortunately bring it to your attention that I am, in fact, NOT verbose.


Re: Error on dub build - Trying Vibe-d for the first time

2020-10-14 Thread H. S. Teoh via Digitalmars-d-learn
On Wed, Oct 14, 2020 at 05:30:37PM +, Andre Pany via Digitalmars-d-learn 
wrote:
> On Wednesday, 14 October 2020 at 16:39:39 UTC, Imperatorn wrote:
> > On Wednesday, 14 October 2020 at 15:27:46 UTC, Andre Pany wrote:
[...]
> > > Please add this to your dub.json file:
> > > "versions": [ "VibeHighEventPriority" ]
> > > 
> > > You may have to restart your pc 1 time. After that the issue
> > > should not occur anymore. (It is a known bug).
[...]
> > Where is this documented?
> 
> I dont know whether it is documented somewhere. It was asked multiple
> times in the forum therefore I remembered.
> 
> Maybe it could be documented in the vibe.d github wiki or on the
> vibe.d website. But hopefully the bug is solved soon.
[...]

Yeah, this is a problem.  Things like these need to be put in the docs
in an easy-to-find way. Like collected in a Troubleshooting page or
something.  If a bug isn't filed yet, I'd file a bug on vibe.d so that
this will get resolved instead of forgotten, and then it will bite the
next newcomer all over again.


T

-- 
Many open minds should be closed for repairs. -- K5 user


Re: Range format specifiers in other languages?

2020-10-12 Thread H. S. Teoh via Digitalmars-d-learn
On Mon, Oct 12, 2020 at 05:51:21AM +, Imperatorn via Digitalmars-d-learn 
wrote:
> On Monday, 12 October 2020 at 00:59:33 UTC, Adam D. Ruppe wrote:
> > On Monday, 12 October 2020 at 00:46:37 UTC, Imperatorn wrote:
> > > To people trying to learn, why is that % before ( needed in the
> > > format string?
> > 
> > The %( ... %) stuff is expanded and repeated for each element inside
> > the given array.
> 
> Thanks, it seems there are some pretty powerful formatting options:
> 
> https://dlang.org/phobos/std_format.html

Indeed.

%(...%) is one of my favorite because it can be nested, so it's a very
useful quick-n-dirty tool for debugging ranges. With .chunks and .map,
you can format just about any range-based data in a one-liner for
dumping debug info.

Another cool one is the `,` digit-grouper:

import std;
void main() {
writefln("%,2d", 1234567890);
writefln("%,3d", 1234567890);
writefln("%,4d", 1234567890);
writefln("%,3?d", '_', 1234567890);
writefln("%,4?d", '\'', 1234567890);
writefln("%,4?.2f", '\'', 1234567890.123);
}

Output:
12,34,56,78,90
1,234,567,890
12,3456,7890
1_234_567_890
12'3456'7890
12'3456'7890.12


T

-- 
Век живи - век учись. А дураком помрёшь.


Re: Why does sum not work in static arrays?

2020-10-11 Thread H. S. Teoh via Digitalmars-d-learn
On Sun, Oct 11, 2020 at 01:26:13PM +, Alaindevos via Digitalmars-d-learn 
wrote:
> Sidenote, sort also not works with static arrays.

Just slice it with [].


T

-- 
I think the conspiracy theorists are out to get us...


Re: question on dub and postgresql

2020-10-07 Thread H. S. Teoh via Digitalmars-d-learn
On Wed, Oct 07, 2020 at 07:15:42PM +, aberba via Digitalmars-d-learn wrote:
[...]
> It seems the D ecosystem is not immediately obvious to some people.
> Dub, compilers, and IDEs are recurring issues.

This is stuff that needs to be documented up-front and in-your-face. For
example, if we're going to be serious about dub, all code examples on
the front page of dlang.org ought to use it.  Invocation syntax should
be shown right there on the front page.  It should permeate all
documentation and frequently alluded to.

Similarly, if we're serious about code.dlang.org, then it also needs to
permeate all code examples on dlang.org throughout.  It needs to be
treated as first-class citizen.  Otherwise, it can only be perceived as
this nebulous side thing of unclear relevance that people will tend to
ignore.


T

-- 
Computerese Irregular Verb Conjugation: I have preferences.  You have biases.  
He/She has prejudices. -- Gene Wirchenko


Re: Efficient sort function allowing own test and swap function as parameter

2020-10-06 Thread H. S. Teoh via Digitalmars-d-learn
On Tue, Oct 06, 2020 at 10:18:39PM +, Alaindevos via Digitalmars-d-learn 
wrote:
> I have a large table consisting of two columns.One with words.Another
> with frequencies. I want to sort them efficiently according to the
> names or frequency.
> For this I need an efficient sort function where I can plugin my
> proper test of order, and proper swap. Currently I do it using an own
> written bubble sort that doesn't scale well.

Why don't you use std.algorithm.sort? That's what the standard library
is for.


T

-- 
Being able to learn is a great learning; being able to unlearn is a greater 
learning.


Re: Accessing non-binary Unicode properties with std.uni

2020-09-29 Thread H. S. Teoh via Digitalmars-d-learn
On Tue, Sep 29, 2020 at 06:14:45PM +, Chloé Kekoa via Digitalmars-d-learn 
wrote:
> On Tuesday, 29 September 2020 at 17:04:51 UTC, H. S. Teoh wrote:
> > OTOH, the relevant Unicode data file that contains East_Asian_Width
> > data (EastAsianWidth.txt) is relatively straightforward to parse.
> > In one of my projects, I wrote a little helper program to parse this
> > file and generate a function that tells me if a given dchar is wide
> > or narrow.
> 
> Thank you. Analyzing the data file seems simple enough. :)

If you're daring, you can try parsing it at compile-time... but in this
case, it's kinda pointless, since the data file doesn't change, so
statically generating the desired code as a separate step seems a more
logical thing to do.


T

-- 
Curiosity kills the cat. Moral: don't be the cat.


Re: Safe to remove AA elements while iterating over it via .byKeyValue?

2020-09-29 Thread H. S. Teoh via Digitalmars-d-learn
On Tue, Sep 29, 2020 at 09:56:41AM +, ikod via Digitalmars-d-learn wrote:
> Hello,
> 
> Sorry if I unintentionally hurt anybody in this thread.
> I'll try to implement sane and correct iteration behavior for AA
> without noticeable performance loss, and propose it if I succeed.

No feelings were hurt, don't worry. :-)  I just wanted to point out that
it's quite a tricky issue, and solving it may not be quite as easy as it
appears.  But if you manage to break new ground in this area, I'd be
very interested to hear!


T

-- 
It's amazing how careful choice of punctuation can leave you hanging:


Re: Accessing non-binary Unicode properties with std.uni

2020-09-29 Thread H. S. Teoh via Digitalmars-d-learn
On Tue, Sep 29, 2020 at 04:22:18PM +, Dukc via Digitalmars-d-learn wrote:
> On Monday, 28 September 2020 at 18:23:43 UTC, Chloé Kekoa wrote:
> > The documentation of std.uni [1] says that the unicode struct
> > provides sets for several binary properties. I am looking for a way
> > to query non-binary properties of a character. Is that possible with
> > std.uni or do I need to use a third-party library?
> > 
> > I am specifically interested in the East_Asian_Width property [2]
> > (which has six allowed values). Trying to access
> > std.uni.unicode.East_Asian_Width results in the error message:
> > 
> > > No unicode set by name East_Asian_Width was found.
> > 
> > [1]: https://dlang.org/library/std/uni.html
> > [2]: https://www.unicode.org/reports/tr11/tr11-38.html
> 
> It seems the East Asian width is Unicode standard 13.0, while Phobos
> implements 6.2. So seems like ca case for a third-party library :(.
[...]

OTOH, the relevant Unicode data file that contains East_Asian_Width data
(EastAsianWidth.txt) is relatively straightforward to parse.  In one of
my projects, I wrote a little helper program to parse this file and
generate a function that tells me if a given dchar is wide or narrow.

Here's the generated function (just copy-n-paste this into your code, no
need for yet another external library dependency):

bool isWide(dchar ch) @safe pure nothrow @nogc
{
if (ch < 63744)
{
if (ch < 12880)
{
if (ch < 11904)
{
if (ch < 4352) return false;
if (ch < 4448) return true;
if (ch == 9001 || ch == 9002) return true;
return false;
}
else if (ch < 12351) return true;
else
{
if (ch < 12353) return false;
if (ch < 12872) return true;
return false;
}
}
else if (ch < 19904) return true;
else
{
if (ch < 43360)
{
if (ch < 19968) return false;
if (ch < 42183) return true;
return false;
}
else if (ch < 43389) return true;
else
{
if (ch < 44032) return false;
if (ch < 55204) return true;
return false;
}
}
}
else if (ch < 64256) return true;
else
{
if (ch < 65504)
{
if (ch < 65072)
{
if (ch < 65040) return false;
if (ch < 65050) return true;
return false;
}
else if (ch < 65132) return true;
else
{
if (ch < 65281) return false;
if (ch < 65377) return true;
return false;
}
}
else if (ch < 65511) return true;
else
{
if (ch < 127488)
{
if (ch == 110592 || ch == 110593) return true;
return false;
}
else if (ch < 127570) return true;
else
{
if (ch < 131072) return false;
if (ch < 262142) return true;
return false;
}
}
}
}

Here's the utility that generated this code:

/**
 * Simple program to parse EastAsianWidth.txt to extract some useful 
info.
 */

import std.algorithm;
import std.conv;
import std.range;
import std.regex;
import std.stdio;

struct CodeRange
{
dchar start, end;

bool overlaps(CodeRange cr)
{
return ((start >= cr.start && start < cr.end) ||
(end >= cr.start && end < cr.end));
}

unittest
{
assert(CodeRange(1,11).overlaps(CodeRange(11,12)));
assert(!CodeRange(1,10).overlaps(CodeRange(11,12)));
}

void merge(CodeRange cr)
{
start = min(start, cr.start);
end = max(end, cr.end);
}

unittest
{
auto cr = CodeRange(10,20);
cr.merge(CodeRange(20,30));

Re: Safe to remove AA elements while iterating over it via .byKeyValue?

2020-09-28 Thread H. S. Teoh via Digitalmars-d-learn
On Mon, Sep 28, 2020 at 08:04:49PM +, ikod via Digitalmars-d-learn wrote:
> On Monday, 28 September 2020 at 19:18:20 UTC, H. S. Teoh wrote:
[...]
> > The problem with arbitrary, unrestricted modification of a container
> > while iterating over it, is that it inevitably leads to
> > counterintuitive results.
> 
> Thanks for your reply, and sorry if I was not clear, but I meant AA.
> AA is unordered container, so for me intuitive behavior for mutations
> visibility during iteration is next:
> 
> 1) you must not see removed keys. Let our keys are (unordered) A D C E
> F, you stay on D and remove E. Then E must not be seen on any future
> iteration steps.
> 
> 2) you may or may not see any inserted keys. As AA is unordered
> container there is no reason to expect anything about position of key
> you just inserted - it can fall before or after current iteration
> position, so implementation is free to show inserted keys or not. I
> prefer not to see new keys.
> 
> 3) you expect to visit all not removed keys exactly once.
> 
> Is it sane?
[...]

It sounds reasonable, but it does constrain your implementation. For
example, (3) is likely to break during a rehash. But not rehashing may
lead to other problems, depending on your hashing implementation.


T

-- 
Time flies like an arrow. Fruit flies like a banana.


Re: Safe to remove AA elements while iterating over it via .byKeyValue?

2020-09-28 Thread H. S. Teoh via Digitalmars-d-learn
On Mon, Sep 28, 2020 at 05:18:41PM +, ikod via Digitalmars-d-learn wrote:
> On Monday, 28 September 2020 at 14:58:15 UTC, Steven Schveighoffer wrote:
[...]
> > One could write a specific function to iterate and remove. I
> 
> This looks like dead end to me, as you may not only remove items from
> arbitrary positions while iterating over collection, but also insert
> items.  Also this can happens not only inside foreach body, but in
> other kinds of iteration. And also there can be nested iterations over
> same collection.

The problem with arbitrary, unrestricted modification of a container
while iterating over it, is that it inevitably leads to counterintuitive
results.

For example, suppose you have a container containing the elements A, B,
C, D, E, and you're iterating over it in that order.

1) Suppose you're at element C, and you decide that you need to insert
F. Then there's the question: should F be included in the current
iteration or not?  What if F sorts before C in the current sort order?
What if F sorts after C?  Should the two cases behave similarly (i.e.,
new elements consistently get included in the current iteration, or they
consistently don't get included in the current iteration)?  What if the
act of inserting F requires an internal reordering of elements in the
container?

2) Suppose you're at element C, and you decide to delete D.
"Obviously", the current iteration should not include D. Or should it?
If you delete A while you're at C, you've already iterated over A, so
there's an inconsistency: deleted elements sometimes get included in the
current iteration, sometimes not.  The problem is, which case is the
"correct" one depends on the user's purpose, which is information that
the container may not have.

Also, what if the act of deleting D requires an internal reorganization
of the container?  If this reorg changes the iteration order, how should
the current iteration proceed? According to the old order, or the new
order?  Note that proceeding with the new order may lead to
counterintuitive results: suppose the new order becomes E -> C -> B ->
A.  That means the current iteration will proceed with B and A, which
have already been encountered before, and skips E.

However, proceeding with the old order may be inefficient: it may
require allocating extra storage to preserve the current iteration order
because the reorganized container no longer has that information.  Or it
may require the container itself to maintain extra information in order
to retain the current iteration order.

3) Suppose you're at element A, and you decide to insert F, with the
resulting order A -> B -> C -> D -> E -> F.  So you proceed as usual.
But at B, you decide to delete F, but that causes the container to
reorganize itself to B -> A -> C -> D -> E.  So in the next iteration
you encounter A again, and insert F again, which causes the container to
reorganize itself to A -> B -> C -> D -> E -> F.  So you get stuck in a
loop between A and B, and termination is no longer guaranteed in
iteration.

Basically, the whole concept of iteration is undermined when the
container can mutate while you're iterating.  It leads to inconsistent
behaviour -- sometimes new elements show up in the current iteration,
sometimes they don't; or some elements may get visited multiple times or
missed altogether -- and iteration may not terminate if your sequence of
operations interacts badly with the container's internal
reorganizations.

Furthermore, trying to work around this by postponing container
reorganizations may lead to other problems, like container inconsistency
(some data structures require immediate reorganization in order to
maintain correctness, etc.).

//

The only way to support container mutation while iterating over it, is
if (1) the allowed operations is constrained (e.g., you can only delete
the current element, not any arbitrary element), and (2) the container
itself must explicitly support such an operation (otherwise deleting the
current element may lead to premature termination of the current
iteration, or errors like dangling pointers or some compromise of
container consistency).

Mixing iteration with arbitrary, unrestricted container mutation is a
road that leads to madness.


T

-- 
Recently, our IT department hired a bug-fix engineer. He used to work for 
Volkswagen.


Re: Array Slicing

2020-09-27 Thread H. S. Teoh via Digitalmars-d-learn
On Sun, Sep 27, 2020 at 01:59:07PM +, DMon via Digitalmars-d-learn wrote:
> Are these in the Specification or Phobos?

See: https://dlang.org/articles/d-array-article.html


T

-- 
Государство делает вид, что платит нам зарплату, а мы делаем вид, что работаем.


Re: Safe to remove AA elements while iterating over it via .byKeyValue?

2020-09-27 Thread H. S. Teoh via Digitalmars-d-learn
On Sun, Sep 27, 2020 at 01:02:04PM +, Per Nordlöw via Digitalmars-d-learn 
wrote:
> Is it safe to remove AA-elements from an `aa` I'm iterating over via
> aa.byKeyValue?

No.  Modifying a container while iterating over it is, in general, a bad
idea (unless the container is designed to be used that way, but even
then, such removal is generally restricted), because it often leads to
highly counterintuitive results.  In the case of AA's, removing an
element may lead to a rehashing, which reorders elements, and your
iteration may miss some elements or repeat some elements or terminate
prematurely. Even without a rehashing, you may encounter inconsistent
behaviours, like some elements going "missing".

You probably want to build a list of keys to remove, then remove them
after the iteration instead.


T

-- 
Why can't you just be a nonconformist like everyone else? -- YHL


Re: Is it possible to "overload" based on visibility?

2020-09-25 Thread H. S. Teoh via Digitalmars-d-learn
On Fri, Sep 25, 2020 at 05:58:08PM +, 60rntogo via Digitalmars-d-learn 
wrote:
> On Friday, 25 September 2020 at 15:21:22 UTC, Steven Schveighoffer wrote:
> > If the input is not ref, you should not return by ref, because then
> > you would be returning a reference to local stack data that is about
> > to be destroyed.
> 
> Yes, I understand that. What I'm really after at this point is that I
> would like to write a clever mixin that would handle all of these
> decisions for me. It should generate a function that takes arguments
> and returns the result by value or const reference depending on what
> is more appropriate for the given types. I was under the impression
> that this could be accomplished using in or some other qualifiers.

You probably need to use the long-form of templates, with separate
function declarations, to accomplish this. E.g.:

template myFunc(Args...) {
static if (shouldReturnByRef!Args)
ref ReturnType myFunc(Args args) {
... // implementation here
}
else // non-ref return
ReturnType myFunc(Args args) {
... // implementation here
}
}

The reason is that `ref` return is an attribute of the *function*, not
the return type, so you can't just use an `auto` return type and have
the compiler infer it.


T

-- 
What doesn't kill me makes me stranger.


Re: what's the best way to convert a sysTime to local machine's time (zone)?

2020-09-24 Thread H. S. Teoh via Digitalmars-d-learn
On Thu, Sep 24, 2020 at 08:36:38PM -0400, James Blachly via Digitalmars-d-learn 
wrote:
> On 9/24/20 6:22 PM, mw wrote:
[...]
> > I'm just wondering what's the best way to convert sysTime to local
> > machine's time (zone)?
[...]
> It is definitely not easy to find.
> 
> https://dlang.org/phobos/std_datetime_systime.html#.SysTime.timezone
> 
> SysTime struct contains member fn timezone which will return the
> current settings' TZ ; you can then apply this TZ to other functions
> in std.datetime to transform datetimes to the TZ [of your choice].

Seriously, this should be put in the std.datetime docs on the first
page. It's a common operation that should not require reading through
multiple non-trivially long doc pages and piecing it together yourself.

Please file a bug against the docs if there isn't already one.


T

-- 
Nearly all men can stand adversity, but if you want to test a man's character, 
give him power. -- Abraham Lincoln


Re: How can I test at compile time whether T is an instance of an interface ?

2020-09-23 Thread H. S. Teoh via Digitalmars-d-learn
On Wed, Sep 23, 2020 at 07:08:47PM +, data pulverizer via 
Digitalmars-d-learn wrote:
> On Wednesday, 23 September 2020 at 18:56:33 UTC, wjoe wrote:
> > 
> > It doesn't occur to me that the compiler doesn't know at compile
> > time that
> > 
> > interface IFoo{}
> > class Foo: IFoo {}
> > 
> > class Foo implements interface IFoo.
> 
> Didn't think that the compiler didn't know but wasn't aware that you
> could use that information to statically dispatch. My mistake, I'll
> shut up now!

Of course the compiler knows. And of course it can use this information
for static dispatch. That's why D is so awesome at metaprogramming. ;-)

What the compiler *doesn't* know is whether a variable of some supertype
of Foo (say Object) implements IFoo, because that's something that can
only be determined at runtime (effectively, you need to downcast to Foo
/ IFoo and test if it's null).  But that's not what the OP is asking for
in this case.


T

-- 
Error: Keyboard not attached. Press F1 to continue. -- Yoon Ha Lee, CONLANG


Re: How can I test at compile time whether T is an instance of an interface ?

2020-09-23 Thread H. S. Teoh via Digitalmars-d-learn
Try this:

interface I {}
class C : I {}
class D {}
struct S {}

pragma(msg, is(C : I)); // true
pragma(msg, is(D : I)); // false
pragma(msg, is(S : I)); // false

So probably what you want is something like this:

void register(C, ARGS...)(ARGS args)
if (behavesLikeFoo!C || is(C : IFoo))
...


T

-- 
Change is inevitable, except from a vending machine.


Re: Why private methods cant be virtual?

2020-09-21 Thread H. S. Teoh via Digitalmars-d-learn
On Mon, Sep 21, 2020 at 07:43:30PM -0400, Steven Schveighoffer via 
Digitalmars-d-learn wrote:
[...]
> No, it's not a bug. It's intentional.
> 
> private and package functions are final, and we aren't going to change
> that now, even if it makes sense to make them virtual.
[...]

Whoa.  But why??  What's the reasoning behind private being non-virtual?


T

-- 
What doesn't kill me makes me stranger.


Re: Why private methods cant be virtual?

2020-09-21 Thread H. S. Teoh via Digitalmars-d-learn
On Mon, Sep 21, 2020 at 11:17:13PM +, claptrap via Digitalmars-d-learn 
wrote:
> Seems like a completely pointless restriction to me.
[...]

It looks like a bug to me.  Please file one if there isn't already one:

https://issues.dlang.org/


T

-- 
Computerese Irregular Verb Conjugation: I have preferences.  You have biases.  
He/She has prejudices. -- Gene Wirchenko


Re: Good repos to learn D

2020-09-19 Thread H. S. Teoh via Digitalmars-d-learn
On Sat, Sep 19, 2020 at 08:26:36AM +, Imperatorn via Digitalmars-d-learn 
wrote:
> What are some good examples of pretty large/medium size, good
> structured repos in D? I'm looking for examples to learn from
[...]

Phobos itself.  I have to say, it's the most readable programming
language standard library that I've come across. I've tried to read
glibc code before, and I will never ever do that again(!). Phobos, by
contrast, is a pleasure to read (except for a small number of dark
corners).


T

-- 
Unix was not designed to stop people from doing stupid things, because that 
would also stop them from doing clever things. -- Doug Gwyn


Re: How to convert member function to free function?

2020-09-18 Thread H. S. Teoh via Digitalmars-d-learn
On Fri, Sep 18, 2020 at 06:20:41PM +, Andrey Zherikov via 
Digitalmars-d-learn wrote:
> How can I rewrite foo() function as a free-function that won't cause
> struct copying? My simplified code is:
> 
> struct S
> {
> ref S foo() return
> {
> return this;
> }
> }
> 
> void main()
> {
> S().foo().foo().foo();
> }
> 
> If I write it like "auto foo(S s) { return s; }" then statement in
> main() will copy value of S three times and I want to avoid this.

Why can't you return by ref, which would also avoid the copying?

ref S foo(return ref S s) { return s; }


T

-- 
Let's not fight disease by killing the patient. -- Sean 'Shaleh' Perry


Re: Proper way to exit with specific exit code?

2020-09-18 Thread H. S. Teoh via Digitalmars-d-learn
On Fri, Sep 18, 2020 at 08:20:59AM +, IGotD- via Digitalmars-d-learn wrote:
> On Friday, 18 September 2020 at 05:02:21 UTC, H. S. Teoh wrote:
> > 
> > That's the obvious solution, except that actually implementing it is not
> > so simple.  When you have multiple threads listening for each other
> > and/or doing work, there is no 100% guaranteed way of cleanly shutting
> > all of them down at the same time.  You can't just clean up the calling
> > thread and leave the others running, because the other threads might
> > hold references to your data, etc..  But there's no universal protocol
> > for shutting down the other threads too -- they could be in a busy loop
> > with some long-running computation, or they may not be checking for
> > thread messages, or they could be in a server loop that is designed to
> > keep running, etc..  It's one of those annoying things that reduce to
> > the halting problem in the general case.
[...]
> I think a pragmatic solution is just to mutex protect the D exit
> function in case several threads tries to use simultaneously. Then if
> more threads call exit, it will do nothing as the first one that
> called exit actually do the tear down.

That does not solve the problem.  If thread 1 calls exit but thread 2 is
still running and processing data via a shared reference with thread 1's
data, you absolutely do not want to run dtors and tear-down code until
thread 2 is done, otherwise you have a problem.

OTOH, waiting for thread 2 to finish first comes with its own problems:
what if thread 2 never calls exit?  Then no cleanup will be done, which
may not be desirable either (maybe you had thread 1 call exit because
you wanted to release unused resources).


> Also, it should be responsibility of the program to ensure that its
> tear down code runs before calling the D exit function. That's the
> only way I can think of because waiting for all other threads to
> release their resources and exit isn't really realistic either as that
> might do that the program exit never happens. Whatever you do you, you
> have to resort to some "manual" solution".
[...]

If you're prepared to do manual teardown, then you do not need a
D-specific exit function. Just call core.sys.stdc.stdlib.exit and call
it a day. :-)


T

-- 
Never trust an operating system you don't have source for! -- Martin Schulze


Re: Proper way to exit with specific exit code?

2020-09-17 Thread H. S. Teoh via Digitalmars-d-learn
On Thu, Sep 17, 2020 at 09:53:07PM -0400, James Blachly via Digitalmars-d-learn 
wrote:
[...]
> I never considered this -- so when I call core.stdc.stdlib : exit,
> none of my destructors get called?

Yes.


> Presumably also not scope(exit) blocks?

Yes.


> If this is the case, could we simply add a publically-accessible
> shutdown hook in the runtime?

That's the obvious solution, except that actually implementing it is not
so simple.  When you have multiple threads listening for each other
and/or doing work, there is no 100% guaranteed way of cleanly shutting
all of them down at the same time.  You can't just clean up the calling
thread and leave the others running, because the other threads might
hold references to your data, etc..  But there's no universal protocol
for shutting down the other threads too -- they could be in a busy loop
with some long-running computation, or they may not be checking for
thread messages, or they could be in a server loop that is designed to
keep running, etc..  It's one of those annoying things that reduce to
the halting problem in the general case.

Unless we adopt some kind of exit protocol that will apply to *all*
threads in *all* D programs, I don't see any way to implement something
that will work in the general case.


T

-- 
If blunt statements had a point, they wouldn't be blunt...


Re: enum and const or immutable ‘variable’ whose value is known at compile time

2020-09-17 Thread H. S. Teoh via Digitalmars-d-learn
On Thu, Sep 17, 2020 at 06:06:56PM -0400, Steven Schveighoffer via 
Digitalmars-d-learn wrote:
> On 9/17/20 9:13 AM, Simen Kjærås wrote:
> 
> > To be clear: I don't mind 'enum' being used this way, but if I were
> > to do things over again, I would have used 'alias'.
> 
> fun fact: for a (very) brief time, D had a `manifest` keyword that did
> exactly what enum does in this instance (not even sure it made it into
> a release).
> 
> enum is a head scratcher of a name, for sure. But it works out just
> fine once you get used to it. I think of it as "compile time". To be
> honest, for what it does, enum is a very poor name. But because it's
> consistent, it works.
[...]

In my mind, I just substitute 'enum' with 'manifest constant' and
everything makes sense. ;-)


T

-- 
What are you when you run out of Monet? Baroque.


Re: Building LDC runtime for a microcontroller

2020-09-17 Thread H. S. Teoh via Digitalmars-d-learn
On Thu, Sep 17, 2020 at 08:40:06PM +, wjoe via Digitalmars-d-learn wrote:
> On Thursday, 17 September 2020 at 19:27:41 UTC, Adam D. Ruppe wrote:
> > fyi my baby was just born i'll come back to this but it might be a
> > day or two
[...]

"A day or two"??!  More likely a month for starters. :-D :-D :-D  If
even that!  (Speaking from experience here. ;-))

Congratulations!


T

-- 
Живёшь только однажды.


Re: Proper way to exit with specific exit code?

2020-09-17 Thread H. S. Teoh via Digitalmars-d-learn
On Thu, Sep 17, 2020 at 02:58:48PM +, drathier via Digitalmars-d-learn 
wrote:
> What's the proper way to exit with a specific exit code?
> 
> I found a bunch of old threads discussing this, making sure
> destructors run and the runtime terminates properly, all of which
> seemingly concluding that it's sad that there isn't a way to do this
> easily, but hopefully things have changed in the last 5-10 years and
> I'm just missing the obvious solution.

AFAIK, there still isn't an "official" way to do this besides return
the exit code from main().  My go-to solution is to declare an
ExitException that's explicitly caught by main():

class ExitException : Exception {
int returnCode;
this() { super("exit"); }
}

void exit(int rc=0) { throw new ExitException(rc); }

int main(string[] args) {
try {
... // your code here
exit(123);
...
} catch (ExitException e) {
return e.returnCode;
}
return 0;
}

Caveat: this may or may not do the Right Thing(tm) in a multithreaded
application.


T

-- 
Talk is cheap. Whining is actually free. -- Lars Wirzenius


Re: Why is BOM required to use unicode in tokens?

2020-09-14 Thread H. S. Teoh via Digitalmars-d-learn
On Mon, Sep 14, 2020 at 09:49:13PM -0400, James Blachly via Digitalmars-d-learn 
wrote:
> I wish to write a function including ∂x and ∂y (these are trivial to
> type with appropriate keyboard shortcuts - alt+d on Mac), but without
> a unicode byte order mark at the beginning of the file, the lexer
> rejects the tokens.
> 
> It is not apparently easy to insert such marks (AFAICT no common tool
> does this specifically), while other languages work fine (i.e., accept
> unicode in their source) without it.
> 
> Is there a downside to at least presuming UTF-8?

Tested it locally, with and without BOM; the lexer rejects ∂ as a valid
token. I suspect the reason has nothing to do with BOMs, but with the
fact that ∂ is not classified as an alphanumeric (see std.uni.isAlpha,
which returns false for ∂).  The following code, which contains Cyrillic
letters, compiles just fine without BOM (std.uni.isAlpha('Ш') returns
true):

void main() {
int Ш = 1;
writeln(Ш);
}

As the docs for std.uni.isAlpha states, it tests for general Unicode
category 'Alphabetic'.  Probably identifiers are restricted to
characters of this category plus the numerics and '_' (and maybe one or
two others, perhaps '$'? Don't remember now).


T

-- 
People say I'm indecisive, but I'm not sure about that. -- YHL, CONLANG


Re: how to do this meta-programming? print the address of random element's address of a variable length of arrays?

2020-09-12 Thread H. S. Teoh via Digitalmars-d-learn
On Sat, Sep 12, 2020 at 07:31:57PM +, mw via Digitalmars-d-learn wrote:
[...]
> I've tried something like this: the AliasSeq specify the logical
> divide
> 
>printRandomElemAddr(AliasSeq!(extraA, extraB, extraC, extraD), a, b, c);
> 
> but cannot make it work.

Yes, because AliasSeq auto-expands, so `func(AliasSeq!(A,B,C),
AliasSeq!(D,E,F))` is equivalent to `func(A,B,C,D,E,F)`.


> (I'm asking for a more general solution, e.g. what if we have 3, or N
> sets of variadic parameters?)
> 
> Looks like we can only pass 1 variadic parameters, then the question
> is what's the best way to divide it?
> 
> Is there any special marker (variable or type?) can be used to divide?
> and what's the staticSplit?

Maybe the nested templates hack could be used? It depends on what
exactly you're trying to do though:

template myFunc(Args1...) {
template myFunc(Args2...) {
auto myFunc(/* runtime params here */) {
// you can use Args1.length and
// Args2.length here.
...
}
}
}

myFunc!(A,B,C)!(D,E,F)(/* runtime args here */);

Alternatively, define a non-eponymous version of AliasSeq that will
allow you to pass multiple lists of compile-time parameters without
auto-expanding and coalescing them:

template WrappedSeq(T...) {
// N.B.: not eponymous
alias members = T;
}

auto myFunc(Args1, Args2)(/*runtime params*/) {
alias list1 = Args1.members; // need manual unwrap
alias list2 = Args2.members;
}

// (Yeah the caller side will look ugly. C'est la vie.)
myFunc!(WrappedSeq!(A,B,C), WrappedSeq!(D,E,F))(/*runtime args*/);


T

-- 
Why can't you just be a nonconformist like everyone else? -- YHL


Re: tupleof seems to break encapsulation

2020-09-04 Thread H. S. Teoh via Digitalmars-d-learn
On Fri, Sep 04, 2020 at 07:36:00PM +0200, Jacob Carlborg via 
Digitalmars-d-learn wrote:
[...]
> It's useful for serialization and, as you can see in your example, for
> debugging as well. `writeln` will print the values of the fields in a
> struct, even for private fields.

It's certainly useful, but violates private.  I'd propose restricting it
to -debug or at the very least @system, so that normal @safe code will
behave as expected.


T

-- 
It's amazing how careful choice of punctuation can leave you hanging:


Re: BetterC + WASM Update

2020-09-03 Thread H. S. Teoh via Digitalmars-d-learn
On Thu, Sep 03, 2020 at 06:18:00PM +, Mike Brown via Digitalmars-d-learn 
wrote:
[...]
> I did however, get taken in by how small and precise the D code was
> compared to my C++ equivalent. The templating is beautiful, I’m now
> considering writing in D rather than C++ as the smaller and clearer
> code would be of real benefit.
[...]

I also came from a C/C++ background, and D's conciseness, beautiful
syntax, and general "hygiene" continue to impress me every day. I gave
up C++ for D years ago, and have never looked back since.


T

-- 
Без труда не выловишь и рыбку из пруда. 


Re: How to create compile-time container?

2020-08-31 Thread H. S. Teoh via Digitalmars-d-learn
On Mon, Aug 31, 2020 at 08:39:10PM +, Andrey Zherikov via 
Digitalmars-d-learn wrote:
> On a low level, I want something like this code to work:
> string[] arr;  // this can be any suitable type
> 
> arr ~= "a";// data is compile-time constant
> 
> enum f = arr[0];   // fails with "Error: variable arr cannot be read at
> compile time"
> enum b = arr[$-1]; // fails with "Error: variable arr cannot be read at
> compile time"
> 
> More generic description: I'm looking for a way to have a container
> (or any other type) that I can manipulate during compile time.
> Operations needed are similar to stack functionality: get first/last
> element, push_back and pop_back.
> 
> How can I do that?

First, read this article:

https://wiki.dlang.org/User:Quickfur/Compile-time_vs._compile-time

Secondly, to answer your question: write regular ("runtime") code for
manipulating your data (also regular "runtime" data), and simply run the
function during CTFE. Almost 90% of the language is usable during CTFE,
so that's all you really need.


T

-- 
Маленькие детки - маленькие бедки.


Re: How to get the element type of an array?

2020-08-25 Thread H. S. Teoh via Digitalmars-d-learn
On Tue, Aug 25, 2020 at 03:02:14PM +, FreeSlave via Digitalmars-d-learn 
wrote:
> On Tuesday, 25 August 2020 at 03:41:06 UTC, Jon Degenhardt wrote:
> > What's the best way to get the element type of an array at compile
> > time?
> > 
> > Something like std.range.ElementType except that works on any array
> > type. There is std.traits.ForeachType, but it wasn't clear if that
> > was the right thing.
> > 
> > --Jon
> 
> Why not just use typeof(a[0])
> 
> It does not matter if array is empty or not. Typeof does not actually
> evaluate its expression, just the type.

+1, why didn't I think of this before. :-D


T

-- 
Political correctness: socially-sanctioned hypocrisy.


Re: How to get the element type of an array?

2020-08-24 Thread H. S. Teoh via Digitalmars-d-learn
On Tue, Aug 25, 2020 at 03:41:06AM +, Jon Degenhardt via 
Digitalmars-d-learn wrote:
> What's the best way to get the element type of an array at compile
> time?
> 
> Something like std.range.ElementType except that works on any array
> type.  There is std.traits.ForeachType, but it wasn't clear if that
> was the right thing.
[...]

alias T = ... /* the type to inspect */
static if (is(T : E[], E)) {
... /* in this block, E is the element type */
}

Or if you need to use it in another expression:

template ArrayElemType(T) {
static if (is(T : E[], E)) {
alias ArrayElemType = E;
}
else
alias ArrayElemType = void;
// or static assert(0) if it should be an error
}

int[] myArr;
pragma(msg, ArrayElemType!(typeof(myArr))); // int

string myStr;
pragma(msg, ArrayElemType!(typeof(myStr))); // immutable(char)


T

-- 
The best way to destroy a cause is to defend it poorly.


Re: Which version of DMD does GDC 10 target

2020-08-19 Thread H. S. Teoh via Digitalmars-d-learn
On Thu, Aug 20, 2020 at 04:28:41AM +, Arun via Digitalmars-d-learn wrote:
> Which version of DMD is GDC 10 based on?

Compile the following D program to find out:

-
static assert(0, "Compiler language version: " ~ __VERSION__.stringof);
-

I have this line in a file called langver.d, and whenever the exact
language version isn't obvious, I compile it to find out the version.
:-)  (And yes it deliberately asserts instead of using pragma(msg) so
that I don't have to type -o- or -of- or -c or whatever to suppress
actual code emission, just `$compiler langver.d`.)


--T


Re: Location of core.internal.traits : CoreUnqual in Dlang GitHub repo

2020-08-19 Thread H. S. Teoh via Digitalmars-d-learn
On Wed, Aug 19, 2020 at 10:47:19PM +, data pulverizer via 
Digitalmars-d-learn wrote:
> Hi all,
> 
> I've been looking for where `CoreUnqual` is defined in the Dlang
> repos. I've tried searching in dmd and phobos but only found the
> reference usage in std.traits:
> 
> import core.internal.traits : CoreUnqual;
> 
> I'd like to know where it is defined so I can (attempt to!) study the
> code.
[...]

core.* is defined in druntime. So it should be in:

druntime/src/core/internal/traits.d


T

-- 
All men are mortal. Socrates is mortal. Therefore all men are Socrates.


Re: Creating a pointer array

2020-08-19 Thread H. S. Teoh via Digitalmars-d-learn
On Wed, Aug 19, 2020 at 08:09:31PM +, bachmeier via Digitalmars-d-learn 
wrote:
[...]
> Maybe I'm the only one, but I think double*[] is hideous, and I'd sure
> hate for someone not used to D to see it.

IMO, double*[] is absolutely logical. It's a natural consequence of type
syntax.


> Alias is your friend. I think this is much nicer:
> 
> void main() {
> alias double* DoublePtr;
> DoublePtr[] arr;
[...]

IMO, this is far too verbose.  If I had 10 arrays of 10 different types
of pointers, I wouldn't want to write 10 aliases just for that. Plus,
hiding things behind an alias means someone who reads your code has to
look up the alias definition to figure out what it means, whereas if
they see double*[] the exact meaning is immediately obvious.

I'd only use an alias for inordinately-long type names, like function
pointers or delegates with lots of parameters, or verbose type names
like const(int[])[string]. (And even in the latter case, only when the
name occurs frequently.)

But as they say, YMMV. :-)


T

-- 
"A man's wife has more power over him than the state has." -- Ralph Emerson


Re: Subtyping with alias this

2020-08-17 Thread H. S. Teoh via Digitalmars-d-learn
On Tue, Aug 18, 2020 at 05:34:58AM +, novice3 via Digitalmars-d-learn wrote:
[...]
> The problem is:
> if i use fmt.spec in overloaded toString(),
> when i get error "phobos/std/format.d(2243): Error: no property ip for
> type onlineapp.IpV4Address"

Here's a working example:

-
import std;

struct IpV4Address
{
private uint ip;
alias ip this;

void toString(W,Char)(W sink, FormatSpec!Char fmt)
{
if (fmt.spec == 's') {
// Deal with %s here
sink.formattedWrite("%d.%d.%d.%d",
(ip >> 24) & 0xFF, (ip >> 16) & 0xFF,
(ip >> 8) & 0xFF, ip & 0xFF);
} else {
// Everything else
formatValue(sink, this.ip, fmt);
}
}

// This unittest is to workaround the sillyness that if toString
// doesn't compile, std.format pretends it doesn't exist and/or
// the compiler swallows the real error and substitutes it with
// an irrelevant error that has nothing to do with the real
// problem.
unittest
{
IpV4Address ip;
auto app = appender!string;
ip.toString(app, FormatSpec!char.init);
}
}

unittest
{
auto ip = IpV4Address(0x01020304);
writefln("%s", ip);
writefln("%x", ip);
writefln("%d", ip);
}
-


T

-- 
No! I'm not in denial!


Re: Subtyping with alias this

2020-08-17 Thread H. S. Teoh via Digitalmars-d-learn
On Mon, Aug 17, 2020 at 02:29:47PM +, novice3 via Digitalmars-d-learn wrote:
[...]
> ```
> struct IpV4Address
> {
>   private uint ip;
>   alias ip this;
> 
>   string toString()
>   {
> import std.conv: to;
> return to!string((ip >>> 24) & 0xFF) ~ "." ~
>to!string((ip >>> 16) & 0xFF) ~ "." ~
>to!string((ip >>> 8) & 0xFF) ~ "." ~
>to!string(ip & 0xFF);
>   }

What you need is to create an overload of toString that takes a
FormatSpec parameter, so that you can decide what should be output for
which format spec.  Something along these lines:

string toString(W,Char)(W sink, FormatSpec!Char fmt) {
if (fmt.spec == "s")
return to!string((ip >>> 24) & 0xFF) ~ "." ~
to!string((ip >>> 16) & 0xFF) ~ "." ~
to!string((ip >>> 8) & 0xFF) ~ "." ~
to!string(ip & 0xFF);
else
// Fallback to usual uint-formatting
return format("%s", ip);
}


T

-- 
We've all heard that a million monkeys banging on a million typewriters
will eventually reproduce the entire works of Shakespeare.  Now, thanks
to the Internet, we know this is not true. -- Robert Wilensk


Re: Types of lambda args

2020-08-16 Thread H. S. Teoh via Digitalmars-d-learn
On Mon, Aug 17, 2020 at 12:20:24AM +, Cecil Ward via Digitalmars-d-learn 
wrote:
> In a lambda, how do we know what types the arguments are? In something
> like
> (x) => x * x

It's implemented as a template, whose argument types are inferred based
on usage context.


> - there I just don’t get it at all. Can you write
> (uint x) => x * x

Of course you can.


> I’m lost.
[...]

If you're ever unsure of what the inferred type(s) are, you can do
replace the lambda with something like this:

(x) { pragma(msg, typeof(x)); return x*x }

which will print out the inferred type when the compiler instantiates
the lambda.


T

-- 
Customer support: the art of getting your clients to pay for your own 
incompetence.


Re: DMD: how to restore old unittest+main

2020-08-13 Thread H. S. Teoh via Digitalmars-d-learn
On Thu, Aug 13, 2020 at 08:30:44AM +, Jonathan via Digitalmars-d-learn 
wrote:
[...]
> Is there a reason you need to run all unittests every time you want to
> run the program?

During development, this is a valuable time-saver: instead of compiling
twice, once with -unittest and once without, then running each
individually, it helps to have unittests run before main() so that any
regressions are immediately noticed, and if unittests pass, then main()
runs and manual testing can proceed.

Obviously, for release builds unittests are useless, so in that case
there's no need to include unittests before running main().


> I personally compile with -unittest to make sure all my unittests
> pass, then recompile without the -unittest flag if I actually want to
> run the program.  This way, time isn't wasted running unittests every
> time the program is run.

Lots of time is wasted if you have to compile twice, once with -unittest
and once without, while you're in the code-compile-test cycle and need
to run main() for manual testing. (Not everything is testable with
unittests!)


T

-- 
"Holy war is an oxymoron." -- Lazarus Long


Re: How does D's templated functions implementation differ from generics in C#/Java?

2020-08-08 Thread H. S. Teoh via Digitalmars-d-learn
On Sat, Aug 08, 2020 at 01:47:27AM +, jmh530 via Digitalmars-d-learn wrote:
> On Friday, 7 August 2020 at 21:39:44 UTC, H. S. Teoh wrote:
> > [snip]
> 
> "Furthermore, it can dispatch to a type-erased implementation ala Java
> -- at your choice;"
> 
> This is interesting. Would you just cast to Object?

You could. Or you could introspect a type and dispatch a subset of types
to a type-erased implementation, say under some base class of your
choice or something. Or even something else altogether, like a C-style
void*-based implementation.

Or a hybrid, like this:

class ContainerBase {
class Node {
Node next, prev;
... // generic methods independent of element type
}
...
}

class Container(T) : ContainerBase {
class Node : ContainerBase.Node {
T payload;
... // T-specific methods
}
...
}

The code in ContainerBase deals with the element-independent part of the
container, essentially the type-erased component, whereas Container!T
retains type information for working with type-specific operations.
Result: reduced template bloat like Java generics, yet without giving up
the advantages of retaining element type information.

You couldn't do such a thing in Java, because Java-style generics simply
aren't powerful enough.


T

-- 
Truth, Sir, is a cow which will give [skeptics] no more milk, and so they are 
gone to milk the bull. -- Sam. Johnson


Re: How does D's templated functions implementation differ from generics in C#/Java?

2020-08-07 Thread H. S. Teoh via Digitalmars-d-learn
On Fri, Aug 07, 2020 at 09:03:47PM +, aberba via Digitalmars-d-learn wrote:
> Syntactically they look the same (although D's can do more things) so
> I'm trying to understand how why in D it's called template but in
> languages like C#/Java they're generics.
> 
> I guess I have fair understanding of D's code generation but isn't it
> same as what what is available in those languages too? How are the
> implementation different?

They are *very* different.

Java generics are based on "type erasure", i.e., at the syntactic level,
containers are parametrized with the element types, but at the
implementation level, the element types are merely "erased" and replaced
with Object (a top type of sorts). There is only one container
instantiation, which is shared across all parametrizations.  I don't
know exactly why this approach was chosen, but my guess is, to avoid the
complexities associated with templates (esp. as seen in C++, which was
the prevailing language with type parametrization when Java generics was
being designed), and to avoid template bloat.  But because of how this
implementation works, Java generics are very limited in a lot of ways
that make them feel straitjacketed once you've gotten used to a more
powerful template system like in C++ or especially D. Since the
container does not retain any information about the type parameter, you
cannot perform any type-specific operations on elements (unless you do
runtime introspection -- and I'm not even sure Java lets you do this),
and you cannot make compile-time decisions based on type properties --
because the single container implementation must be able to handle all
type arguments.

D templates do not type-erase, and the generated code retains full
knowledge about the type parameters. Therefore, you can do very powerful
things with them, like Design by Introspection, performing type-specific
operations, generate different code depending on type properties, etc..
Since each template instantiation is distinct, it has the flexibility of
doing completely different things depending on the type arguments,
independently of any other instantiation of the same template.
Furthermore, it can dispatch to a type-erased implementation ala Java --
at your choice; and it can even conditionally do so by inspecting the
properties of the type arguments. IOW, it is a strict superset of Java
generics.

Unfortunately, the power of D templates does come at a cost: if used
carelessly, it can result in a lot of template bloat. Reducing this
bloat often requires delicate code surgery or restriction on some of the
flexibility. (Though IMO, this is not a bad thing -- the user is given
the *choice* to use a type-erased implementation if he so chooses, or
control the template bloat in other ways; in Java, you have no choice
but to live with the limitations of a type-erased generics system. But
then again, Java has always been a bondage-and-discipline kind of
language, so this isn't anything unexpected. People just learn to live
with it.)


T

-- 
Time flies like an arrow. Fruit flies like a banana.


Re: I just discovered an alternative use of the `in`-operator

2020-08-07 Thread H. S. Teoh via Digitalmars-d-learn
On Fri, Aug 07, 2020 at 09:02:03PM +, Per Nordlöw via Digitalmars-d-learn 
wrote:
> On Thursday, 6 August 2020 at 22:24:43 UTC, Paul Backus wrote:
> > [1] https://dlang.org/spec/expression.html#is_expression
> 
> I bet there a several places in Phobos where this feature isn't but
> could be used.

Perhaps. But as I recall, it's widely used throughout Phobos, so it
probably won't be easy to find a place where it could be used but isn't.


T

-- 
Written on the window of a clothing store: No shirt, no shoes, no service.


Re: Non-recursive maxSizeOf

2020-08-06 Thread H. S. Teoh via Digitalmars-d-learn
On Thu, Aug 06, 2020 at 01:07:14PM +, Per Nordlöw via Digitalmars-d-learn 
wrote:
[...]
> However, your solution
> 
> template maxSize(T...)
> {
> align(1) union Impl { T t; }
>   enum maxSize = Impl.sizeof;
> }
> 
> fails as
> 
> variable `std.variant.maxSize!(void, string).Impl.__t_field_0`
> variables cannot be of type `void`
> 
> because one of the instances of `maxSize` in std.variant has `void` as
> a member `T...`.

Ugh, apparently `void` is one of those nasty inconsistent things that
has all sorts of ad hoc behaviour:

- void cannot be used to instantiate variables (behaves sortof like a
  bottom type);
- But in spite of that, void.sizeof == 1, rather than the expected 0. My
  guess is that this has to do with the hack to make the size of void[]
  equal to its length;
- void in void[] is a top type (but only in that context -- but its
  .sizeof is inconsistent with this);
- void is a unit type when specified as a function return type (but in
  spite of that cannot be instantiated, contrary to a true unit type,
  and this contradicts its being a top type in void[]);

IOW, void is a mishmash of ad hoc things thrown together in a way that
isn't even self-consistent. A chimera of exceptions that, it would seem,
always require special-casing to handle.


> Do you have any simple solution to this, H. S. Teoh?

Unfortunately, no. :-(  Why does std.variant even support storing void
in the first place?!


T

-- 
People tell me that I'm paranoid, but they're just out to get me.


Re: Non-recursive maxSizeOf

2020-08-06 Thread H. S. Teoh via Digitalmars-d-learn

On Thursday, 6 August 2020 at 00:58:39 UTC, Per Nordlöw wrote:

Is it possible to implement

template maxSizeOf(T...)
{
static if (T.length == 1)
enum size_t maxSizeOf = T[0].sizeof;
else
{
enum size_t firstSize = T[0].sizeof;
enum size_t maxSizeRest = maxSizeOf!(T[1 .. $]);
enum size_t maxSizeOf = firstSize >= maxSizeRest ? 
firstSize : maxSizeRest;

}
}

in a non-recursive way?


Of course. In fact, it's trivial:

--
template maxSizeOf(T...)
{
align(1) union Impl {
T t;
}
enum maxSizeOf = Impl.sizeof;
}
--

To check that this does what we expect:
--
pragma(msg, maxSizeOf!(char)); // 1LU
pragma(msg, maxSizeOf!(char, short, ubyte)); // 2LU
pragma(msg, maxSizeOf!(char, long, ubyte)); // 8LU

align(1) struct S {
long l;
ubyte b;
}
pragma(msg, maxSizeOf!(long, S)); // 9LU
pragma(msg, maxSizeOf!(int, ubyte[7])); // 7LU
pragma(msg, maxSizeOf!(int, ubyte[3])); // 4LU
--

Why bother with recursion and all of that fancy gobbledygook when 
the compiler already knows how to compute the maximum size of 
something? ;-)


Re: Invoking the compiler during runtime

2020-08-05 Thread H. S. Teoh via Digitalmars-d-learn
On Wed, Aug 05, 2020 at 06:02:58AM +, cy via Digitalmars-d-learn wrote:
> D's compile-time-execution is fantastic, but there are some times when
> I'd like to examine the generated code, or produce code that needs to
> pass through earlier phases before CTFE, or do AST stuff. Sometimes I
> simply don't want to generate the code with every compilation, so
> saving the generated code in a file some would be really neat, if I
> could then invoke the compiler during runtime to build it. Plus
> backtraces wouldn't just be "dlang-mixin-397."

TBH, in spite of D's amazing metaprogramming capabilities, there comes a
point where compile-time generated code just becomes too unwieldy. Like
if you have multi-page mixin strings, or mixins nested to the n'th
level, or your token strings span 10+ pages and it becomes just
impossible to debug.

In those cases, my standard solution is to write an auxiliary program
that emits D code, and then just compile that with the main program as a
second step. The nice thing about that is, as you said, you can examine
the generated code, run coverage on it or whatever other analysis, or
just plain debug the generating code until the output looks like it
might compile before you actually try to compile it, instead of dealing
with multi-page template instantiation errors that only a Klingon could
decipher.


[...]
> In C I write a utility program in awful C to print out C code, have a
> cmake custom command run that awful C as needed, and assume the
> existence of that generated source in my main program.

That's exactly what I'd do.  The utility program, of course, would also
be in D. :-D  With unittests, even, that ensure the output is
syntactically correct. :-P  How's that for self-reference? :-P


> So is that the best way to do it? Have a side program that writes D
> source to a file, and then the main program simply imports the file?

Yep.


> Or is there some AST syntax I should be generating instead? Some way
> to import the compiler itself, instead of calling it in a subprocess?

IMO it's not worth the trouble to import the compiler itself, unless you
want to do runtime dynamic (re)compilation.  I've also done that, but
generally I advise against importing the compiler, but instead just run
it as a separate process (say using std.process, which is very
convenient).  Generate the code you want to compile in string form, spit
it to a temporary file and compile it (or pipe it through stdin to `dmd
-`) into a shared object, then load it dl_open on Posix (or whatever the
Windows equivalent it) and run the code that way.  Before the recent
fiasco, dmd is fast enough that generally you won't notice the lag
unless you do this in a tight loop.

But I wouldn't recommend this unless you want to do it at runtime, since
you'll have to ship a copy of dmd with your program, which comes with
its own bag o' worms (you essentially need to ship an entire working
installation of dmd for this to work).  If you just need to generate
some complicated D code at build time, just write a helper utility to
emit D code, and call it a day.

I have a project where I need to generate D code from large data files
that would be too resource-intensive to do from inside CTFE, so a
utility that emits D is what I went with.

In the same project I also have a helper program that scans GLSL (vertex
shader) code and generates D APIs for each shader (automates binding
parameters, generation of wrapper functions, etc.).  Possible to do with
CTFE if I tried hard enough, but at the end of the day, why would I?
Doing it in CTFE bloats compile-time beyond my patience, and requires
onerous resources, and besides, CTFE cannot access OS functions like
scanning directories; so why not just write a helper D program that has
full OS access, runs faster, and does what it needs to do to generate D
code that I can then import in my main program.


> Is there a way in dub to specify that you run a D program contained in
> X.d during the build process, to build Y.d, that Z.d imports?

I don't know if the latest version of dub can do that, but the last time
I checked about a year or so ago, it couldn't. Or at least, not without
jumping through hoops and bending over backwards.  But that's not a
problem for me anyway, since I use SCons to build my projects, and SCons
has no problem at all building multiple helper programs that each
generate some subset of D source files, and then compiling said files
into the final product. In fact, even multiple final products (in the
aforementioned project, for example, my SCons script builds both an APK
for Android with an LDC cross-compiler, and a Linux executable for on-PC
testing at the same time with dmd :-P).  This method of building
generally goes against the grain of dub's design, though, so instead of
fighting with dub in an uphill battle, I just went with something more
flexible instead.


T

-- 
Windows 95 was a joke, and Windows 98 was the punchline.


Re: Question about UDAs

2020-08-03 Thread H. S. Teoh via Digitalmars-d-learn
On Mon, Aug 03, 2020 at 08:16:57PM -0700, Ali Çehreli via Digitalmars-d-learn 
wrote:
[...]
> UDAs were added to D by a request from Manu Evans and that's when I
> learned them. In one of Manu's use cases they would put a @Tweakable
> attribute to certain struct members. The effect of that attribute
> would be to compile special code that would expose that member in a
> dialog box where the developer would "tweak" its value to see how the
> program (a game) would behave at specific values of that member.
> 
> The awesomeness comes from the fact that once they have this
> @Tweakable machinery, they don't change their code at all: They put
> that attribute to certain members during development, find good values
> and then remove it; perhaps in half an hour. The only addition to code
> is one @Tweakable attribute and some magic produces a dialog box; then
> they remove the attribute. Pretty cool. :)

I've also used it for generating getopt-like code for parsing
command-line parameters.  You might have several subsystems in your
program, each of which comes with a set of parameters that can be
configured; instead of sprinkling this information across multiple
places (once in the subsystem to read the values, once in the call to
getopt to parse the option, once in the code for display detailed
description of the setting, once in configuration file parsing code to
basically do the same thing as getopt except with a config file, ad
nauseaum), just create a struct that contains the parameters for each
subsystem, then use UDAs to decorate each setting with command-line
option name, help text, value ranges, or even custom parsing functions
if you want to get fancy. Then write a generic option-parsing function
that introspects the UDAs to generate help text, option names, default
values, precedences, etc.. Another generic function for parsing the
config file.

Then the next time you want to add a setting, it's just a matter of
adding another field to your struct, tag it with the appropriate UDAs,
and it will "magically" appear in your command-line, config file
parsing, and in-program help text without further ado.

Best of all, once you build this infrastructure, you can easily reuse it
across different programs: the getopt wrapper, help text generator,
config file parser are all completely driven by introspection and UDAs,
so there's nothing program-specific about them. Just copy-n-paste them
into another project, and create your structs, and you're all set. :-)


T

-- 
"I'm not childish; I'm just in touch with the child within!" - RL


Re: Question about UDAs

2020-08-03 Thread H. S. Teoh via Digitalmars-d-learn
On Mon, Aug 03, 2020 at 03:00:08AM +, Cecil Ward via Digitalmars-d-learn 
wrote:
> When practically speaking would you use UDAs? A real-world use-case?

There are probably more use cases than this, but for me, their primary
usefulness is in declarative programming and compile-time introspection.

Here's one specific use case: serialization.  I have a bunch of structs
and classes that I want to serialize, and instead of writing tons and
tons of boilerplate, I use __traits(allMembers) to introspect a generic
type T and generate serialization code for it.  But sometimes some
fields should not be serialized (e.g., they are transients like caches
and stuff).  Or sometimes certain fields may need special treatment,
like a different serialization method depending on domain-specific
information about their contents.

I *could* hard-code this knowledge into the serialization code, but
UDAs provide a better alternative: I tag my types with various UDAs
recognized by the serialization system, so that when it encounters, say,
a string tagged @hexDigits, it knows that it can use a more compact
representation by parsing the string into binary and storing it as a
compact blob, for example.  Or if a field should not be serialized, I'd
tag it @dontSerialize and the serialization code skips over it. By using
UDAs instead of hard-coding into the serialization code, the
serialization can be made generic and reusable across projects.

Other use cases include automatically creating database schemas based on
types: like a bunch of structs representing records, and UDAs to tag
which fields should be indexed, which fields have constraints, etc..
Then the database backend code can just introspect these types and
automatically generate schemas, query code, etc..

Since UDAs can be arbitrary types, it actually has a lot of uses. For
example, you can use them to inject code for processing data, e.g., by
using a struct as UDA with a method that takes the data and performs
some operation on it. The generic code that reads the UDA can then pass
the data to the method in a completely agnostic way that lets you
separate concerns very cleanly.


T

-- 
In a world without fences, who needs Windows and Gates? -- Christian Surchi


Re: Gotcha with const std.range.chunks

2020-07-30 Thread H. S. Teoh via Digitalmars-d-learn
On Thu, Jul 30, 2020 at 06:52:42PM +, sportsracer via Digitalmars-d-learn 
wrote:
[...]
> int[] xs = new int[100];
> const chunked = xs.chunks(10);
> writeln(chunked[0][0]);
> }
> 
> Error: mutable method std.range.Chunks!(int[]).Chunks.opIndex is not
> callable using a const object

In order to iterate the chunks, you must be able to mutate the range
object returned by .chunks. If it's const, then it's not mutable, and
therefore it cannot be iterated.

If you want to protect the underlying data from mutation, create a const
view of the data first, e.g., something like this:

int[] xs = new int[100];
const(int)[] constView = xs;
auto chunked = constView.chunks(10); // N.B. mutable

The range itself will be mutable, but the elements will be const and not
mutable through the range.


T

-- 
Why can't you just be a nonconformist like everyone else? -- YHL


Re: Overload comparison operators for "nullable" types

2020-07-30 Thread H. S. Teoh via Digitalmars-d-learn
On Thu, Jul 30, 2020 at 01:41:05PM +, Oleg B via Digitalmars-d-learn wrote:
[...]
> Logically we can compare versions, but what must return `opCmp` if one of
> versions has 'not comparible' state?
[...]

opCmp is allowed to return float; so you could return float.nan in this
case.


T

-- 
"Real programmers can write assembly code in any language. :-)" -- Larry Wall


Re: Contributing to D wiki

2020-07-27 Thread H. S. Teoh via Digitalmars-d-learn
On Mon, Jul 27, 2020 at 11:39:32AM +, John Burton via Digitalmars-d-learn 
wrote:
[...]
> I tried looking there for information and examples of getting glfw3
> statically linked into my program using LDC and didn't really find
> anything.
> 
> I wonder if adding a page for static linking tips would be useful as
> it seems to be problematic and compiler and environment dependent?
> Perhaps I should go ahead and see if I can make a page and see if
> anyone objects :P

Yes, please go ahead.  I don't think there's a need to ask. :-)


T

-- 
EMACS = Extremely Massive And Cumbersome System


Re: Help with Ranges

2020-07-27 Thread H. S. Teoh via Digitalmars-d-learn
On Sun, Jul 26, 2020 at 07:10:41AM +, Charles via Digitalmars-d-learn wrote:
> Suppose I have the following line of code where arr is an array,
> doSomething is some predicate that does a lot of processing on each
> element, sort must come after the mapping, and there are more
> operations done to the range after sort:
> 
> arr.map!doSomething.sort. ...;
[...]

As Steven said, you cannot sort a range that doesn't support swapping
elements, and most ranges cannot unless they're backed by actual
storage, like an array. (*Something* has got to keep track of where the
elements are, after all.)

In this particular case, though, if the contents of your original
doesn't need to be preserved, perhaps .schwartzSort might be what you're
looking for?

If you cannot modify the original array for whatever reason, then an
allocation is probably unavoidable -- either you'll have to create an
array of your mapped elements, or you could create an index and sort
that instead (see .makeIndex or std.range.zip for different approaches).


T

-- 
Those who don't understand Unix are condemned to reinvent it, poorly.


Re: std.process - avoid interaction with parent shell

2020-07-20 Thread H. S. Teoh via Digitalmars-d-learn
On Mon, Jul 20, 2020 at 04:55:52PM -0400, Steven Schveighoffer via 
Digitalmars-d-learn wrote:
> I am doing some scripting via D, and using std.process.execute to git
> clone things.
> 
> I don't want any user interaction. Occasionally, I get a repository
> that no longer exists (404). Then git comes up and asks for a
> username/password. I want it to just fail. Apparently git has no
> option to be non-interactive, it supposedly checks stdin to see if
> it's a tty, and only errors if it's not.

Try --no-pager perhaps?  Not sure if that would help, since this isn't
technically a pager that's prompting you.

Another way is to take a look at std.process.execute's implementation. I
believe it's just a wrapper around spawnProcess. What you want is to
adapt that implementation so that it closes stdin before fork-n-exec'ing
git; that should stop any prompts.

One thing to be aware of is that it may not necessarily be git itself
that's prompting you; it could be a helper program like a password
manager that creates the prompt. In that case you probably have to find
out what it is, and disable it somehow (usually by overriding some
environment variable that gets passed to the git child process).


T

-- 
Ph.D. = Permanent head Damage


Re: Good way to send/receive UDP packets?

2020-07-18 Thread H. S. Teoh via Digitalmars-d-learn
On Sat, Jul 18, 2020 at 04:00:09PM +, Dukc via Digitalmars-d-learn wrote:
> I have a project where I need to take and send UDP packets over the
> Internet. Only raw UDP - my application uses packets directly, with
> their starting `[0x5a, packet.length.to!ubyte]` included. And only
> communication with a single address, no need to communicate with
> multiple clients concurrently.
> 
> I understand that I could do it either with the Curl library bundled
> with Phobos, or use Vibe.D or Hunt instead. But it's the first time
> I'm dealing with low-level networking like this, and my knowledge
> about it is lacking.  So seek opinions about what library I should
> use, and more importantly, why.

If you already have the raw packets, there is no need for any library,
just call the OS's C API directly (such as core.sys.posix.sys.socket).
For UDP you don't even need to set up anything, just create a socket and
fire the packets away.


--T


Re: Contributing to D wiki

2020-07-15 Thread H. S. Teoh via Digitalmars-d-learn
On Wed, Jul 15, 2020 at 09:27:22PM +, tastyminerals via Digitalmars-d-learn 
wrote:
[...]
> D wiki is badly outdated. This is not a fact but a gut feeling after
> reading through some of its pages. I was wondering who's owning it
> myself but never actually dared to just go and update.

Why not?  It's a *wiki*.  Wikis are intended for the user community
(i.e. you & me) to go and edit.  That's the whole point of a wiki.  If
that wasn't the intention we wouldn't have set up a wiki in the first
place.


> I just had a feeling it's abandoned.  On the other hand why would it
> be?

It's only as abandoned as this community abandons it.  If people start
updating it and fixing things when they see it instead, then it will not
be abandoned.


T

-- 
There are 10 kinds of people in the world: those who can count in binary, and 
those who can't.


Re: Contributing to D wiki

2020-07-15 Thread H. S. Teoh via Digitalmars-d-learn
On Wed, Jul 15, 2020 at 04:04:56PM +, aberba via Digitalmars-d-learn wrote:
> So I'm looking to make changes to the D wiki but I'm not sure who to
> talk to about such changes.
> 
> Currently: Move all other IDEs low-quality down (maybe to Others) and
> focus on just the few that really works (IntelliJ, Visual Studio Code
> and Visual Studio). Instead of many options that don't work, why not
> focus on they few that works?
[...]

Why not just go ahead and take charge of that part of the wiki?  It's a
wiki after all.


T

-- 
Talk is cheap. Whining is actually free. -- Lars Wirzenius


Re: GDC and DMD incompatability, can both be used?

2020-07-10 Thread H. S. Teoh via Digitalmars-d-learn
On Sat, Jul 11, 2020 at 04:28:32AM +, cy via Digitalmars-d-learn wrote:
> hunt/source/hunt/serialization/JsonSerializer.d:125:20: error: basic
> type expected, not foreach
>   125 | static foreach (string member; FieldNameTuple!T) {
> 
> I'm having a little trouble using the hunt library with gdc. Does gdc
> not support static foreach at all? Is there some way to write code
> that it can understand, which does the same thing?

Which version of GDC are you using?  Static foreach is a relatively new
feature, and GDC may not have picked it up yet.

GDC uses exactly the same front end as DMD (and LDC), so there is no
fundamental incompatibility. The only issue is that GDC is tied to the
GCC release cycle, so it may be a few releases behind DMD, and therefore
lack some newer features.  Eventually it will catch up, though.


T

-- 
When solving a problem, take care that you do not become part of the problem.


Re: What's the point of static arrays ?

2020-07-09 Thread H. S. Teoh via Digitalmars-d-learn
On Thu, Jul 09, 2020 at 06:02:02PM +, IGotD- via Digitalmars-d-learn wrote:
[...]
> Is this the reason that ubyte[arraySize] doesn't create a dynamic
> array with size arraySize?
> 
> Now you need to do.
> 
> ubyte[] arr;
> arr.length = arraySize;

Nah, just do this:

arr = new ubyte[arraySize];

Mission accomplished. ;-)


T

-- 
I am Ohm of Borg. Resistance is voltage over current.


Re: Working with pointers/adresses

2020-07-09 Thread H. S. Teoh via Digitalmars-d-learn
On Thu, Jul 09, 2020 at 05:24:33PM +, matheus via Digitalmars-d-learn wrote:
[...]
> I wonder if the question was more like intended to display the value
> using pointers, ie:
> 
> import std.stdio;
> 
> void main(){
> int i;
> readf("%d\n", i);
> int *p = 
> writeln(*p);
> }
> 
> Because accessing arbitrary memory location seems very weird.
[...]

Unless the goal was to write an OS? ;-)


T

-- 
It only takes one twig to burn down a forest.


Re: Working with pointers/adresses

2020-07-09 Thread H. S. Teoh via Digitalmars-d-learn
On Thu, Jul 09, 2020 at 04:16:53PM +, Quantium via Digitalmars-d-learn 
wrote:
> I have one more question. I tested the programm, as you said, it
> worked rarely because of OS restrictions. If I compile that code to
> dll, and run it through dll launcher, should it work?

No, it's subject to the same restrictions.  Why should it be otherwise?


T

-- 
Let's call it an accidental feature. -- Larry Wall


Re: What's the point of static arrays ?

2020-07-09 Thread H. S. Teoh via Digitalmars-d-learn
On Thu, Jul 09, 2020 at 12:12:06PM +, wjoe via Digitalmars-d-learn wrote:
> + The size is known at compile time.
> 
> vs.
> 
> - Can neither grow nor shrink them
> - Can't append elements
> - Can't remove elements

Consider a 3D game in which you represent vectors as static arrays of 4
elements (homogenous representation).  In this case, it's a plus to not
be able to append/remove elements, because the rest of the code expects
vectors that are exactly 4 elements long.


> - Can't slice them
> - Can't range them

Sure you can.


> - Assignment copies the whole array, as in int[5] a; auto b = a;

Sometimes this is desirable.  Consider the 3D game example.  Suppose
you're given a vector and need to perform some computation on it. If it
were a dynamic array, you'd need to allocate a new array on the heap in
order to work on it without changing the original vector. With a static
array, it's passed by value to your function, so you just do what you
need to do with it, and when you're done, either discard it (== no work
because it's allocated on the stack) or return it (== return on the
stack, no allocations).


> - Size is limited by stack
> - Stack overflow issues

You *can* allocate static arrays on the heap, in which case they become
closer to dynamic arrays.  Generally, the cases for which static arrays
are useful are when the size isn't huge; for huge arrays it makes more
sense to just use dynamic arrays.

You can also have classes that contain large static arrays: in this
case, the usefulness comes from having the array embedded in the class
object itself, rather than being a separate heap allocation + another
level of indirection.


[...]
> Considering the many downsides why would I ever want to choose a
> static over a dynamic array ?

It depends on your needs.  If you know you're always going to be trading
in vectors of a fixed size, like 4-element vectors in aforementioned 3D
game, then there's no need to put extra GC (or whatever allocator)
pressure on your code, just trade in T[4].  You also skip bounds checks
in many cases, since the length is known at compile-time.  You get
by-value semantics, which is generally much easier to reason about than
by-reference semantics.  You avoid an extra level of indirection, which
can matter in hotspots.

If your arrays are going to change in length, then static arrays aren't
what you need; just use dynamic arrays. They serve different purposes.


T

-- 
All men are mortal. Socrates is mortal. Therefore all men are Socrates.


Re: How to ensure template function can be processed during compile time

2020-07-08 Thread H. S. Teoh via Digitalmars-d-learn
On Wed, Jul 08, 2020 at 08:11:05PM +, IGotD- via Digitalmars-d-learn wrote:
[...]
> Doing the same in D, would with my lack of knowledge look like this.
> 
> 
> size_t mySize(T)()
> {
> return T.sizeof + 42;
> }

What you want is:

enum mySize(T) = T.sizeof + 42;

And there is no need for a const overload.


[...]
> int v;
> 
> enum sz = mySize!int // works, returns 46
> enum sz2 = mySize(v) // doesn't work. Error: variable v cannot be read at
> compile time

Yes, because you're trying to pass the value of a variable to mySize,
and that variable doesn't have a value until runtime.


> Here we have a difference between C++ and D as C++ was able infer the
> size of v during compile time.
> 
> Now since mySize is a template, shouldn't this work mySize!v, but it
> doesn't? What essential understanding have I missed here?

https://wiki.dlang.org/User:Quickfur/Compile-time_vs._compile-time


T

-- 
The diminished 7th chord is the most flexible and fear-instilling chord.
Use it often, use it unsparingly, to subdue your listeners into
submission!


Re: Program exited with code -11 when calling

2020-06-30 Thread H. S. Teoh via Digitalmars-d-learn
On Wed, Jul 01, 2020 at 05:04:28AM +, Anthony via Digitalmars-d-learn wrote:
[...]
> auto str_utf8 = str.toUTF8();
> bson_error_t error
> 
> auto bson = bson_new_from_json(cast(const uint8_t*)str_utf8.ptr, -1,
> );
> 
> 
> I get a "Program exited with code -11" message.
> Does anyone know what I'm doing wrong?

D strings are generally not null-terminated (except for literals).
Before passing them to a C function you need to add a trailing null. Try
using std.conv.toStringz instead of casting the pointer yourself.


T

-- 
A programming language should be a toolbox for the programmer to draw upon, not 
a minefield of dangerous explosives that you have to very carefully avoid 
touching in the wrong way.


Re: Why is this allowed

2020-06-30 Thread H. S. Teoh via Digitalmars-d-learn
On Tue, Jun 30, 2020 at 02:06:13PM -0400, Steven Schveighoffer via 
Digitalmars-d-learn wrote:
> On 6/30/20 12:37 PM, Steven Schveighoffer wrote:
[...]
> I take it back, I didn't realize this wasn't something that happened
> with dynamic arrays:
> 
> int[] dyn = [1, 2, 3];
> 
> dyn = 5; // error
> dyn[] = 5; // ok, brackets required
> 
> I would say that's a decent precedent to deprecate and remove that
> functionality.
[...]

Could you add this info to the bugzilla issue as additional evidence
that this (mis)feature should be removed?

https://issues.dlang.org/show_bug.cgi?id=15932


T

-- 
"640K ought to be enough" -- Bill G. (allegedly), 1984. "The Internet is not a 
primary goal for PC usage" -- Bill G., 1995. "Linux has no impact on 
Microsoft's strategy" -- Bill G., 1999.


Re: Why is this allowed

2020-06-30 Thread H. S. Teoh via Digitalmars-d-learn
On Tue, Jun 30, 2020 at 04:50:07PM +, Adam D. Ruppe via Digitalmars-d-learn 
wrote:
> On Tuesday, 30 June 2020 at 16:41:50 UTC, JN wrote:
> > I like my code to be explicit, even at a cost of some extra typing,
> > rather than get bitten by some unexpected implicit behavior.
> 
> I agree, I think ALL implicit slicing of static arrays are problematic
> and should be removed. If you want to set it all or slice it for any
> other reason, just put on the [].

We've known this for at least 4 years, yet nothing has been done about
it. :-(

https://issues.dlang.org/show_bug.cgi?id=15932

The problem isn't just safety, which is partly addressed by dip1000, but
also mistakes of the kind the OP posted.  Basically, it's a misfeature
with many ill side-effects all in the questionable name of convenience.
It's time we got rid of it.


T

-- 
Truth, Sir, is a cow which will give [skeptics] no more milk, and so they are 
gone to milk the bull. -- Sam. Johnson


Re: Why is this allowed

2020-06-30 Thread H. S. Teoh via Digitalmars-d-learn
On Tue, Jun 30, 2020 at 04:22:57PM +, JN via Digitalmars-d-learn wrote:
> Spent some time debugging because I didn't notice it at first,
> essentially something like this:
> 
> int[3] foo = [1, 2, 3];
> foo = 5;
> writeln(foo);   // 5, 5, 5
> 
> Why does such code compile? I don't think this should be permitted,
> because it's easy to make a mistake (when you wanted foo[index] but
> forgot the []).  If someone wants to assign a value to every element
> they could do foo[] = 5; instead which is explicit.

File a bug?

I suspect that one potential reason is that nasty misfeature of static
arrays implicitly converting to a slice of itself, so `foo = 5;` is in
some sense being translated as `foo[] = 5;`.

(And on that note, this implicit static -> dynamic array conversion is
seriously a nasty misfeature that ought to be killed with fire. It leads
to bugs like this:

struct Database {
int[] data;
void set(int[] _data) {
data = _data;
}
}
void myFunc(ref Database db) {
int[3] x;
db.set(x);  // oops
}
)


T

-- 
Once the bikeshed is up for painting, the rainbow won't suffice. -- Andrei 
Alexandrescu


Re: Recursive delegate inside template?

2020-06-26 Thread H. S. Teoh via Digitalmars-d-learn
On Fri, Jun 26, 2020 at 02:12:00PM +, drathier via Digitalmars-d-learn 
wrote:
> I'm trying to get this to compile, without much luck:
> 
> ```
> bool foo()
> {
> template fn(A)
> {
> A delegate(A) fn;
> fn = delegate A(A a)
> {
> return fn(a);
> };
> }
> 
> return fn!(bool)(true);
> }
> ```
> 
> What am I missing here? How can I define a recursive delegate inside
> another function like this?
[...]

Sounds like you're looking for the equivalent of the Y combinator,
perhaps?

https://en.wikipedia.org/wiki/Fixed-point_combinator#Y_combinator


T

-- 
Never trust an operating system you don't have source for! -- Martin Schulze


Re: Unused template arguments; what type to use?

2020-06-26 Thread H. S. Teoh via Digitalmars-d-learn
On Fri, Jun 26, 2020 at 01:40:57PM +, Simen Kjærås via Digitalmars-d-learn 
wrote:
> On Friday, 26 June 2020 at 13:21:25 UTC, drathier wrote:
> > How can I tell the compiler that I will never create a value of type
> > X, while still being able to write code that uses it? Using void as
> > a template parameter is where I started, but I still need to be able
> > to declare variables inside this unreachable function, like `T foo;`
> > even when `T == void`. Can I get any closer to what I want than an
> > empty struct?
> 
> Depends on what you care about, I guess. A final abstract class has
> been my go-to on a few occasions. I'd argue the empty struct is better
> in most cases, and a forward-declared struct with no implementation
> might work in some cases.
[...]

You can also use void[0], which has size 0, unlike an empty struct which
has size 1.


T

-- 
The best compiler is between your ears. -- Michael Abrash


Re: figure out where a particular template function is located

2020-06-24 Thread H. S. Teoh via Digitalmars-d-learn
On Wed, Jun 24, 2020 at 04:28:24PM -0400, Steven Schveighoffer via 
Digitalmars-d-learn wrote:
> I have code that instantiates a template:
> 
> templ!int("abc");
> 
> When I read the source of where I *think* this template should be, I
> can't find one that would match (I think). I feel like it's being
> imported elsewhere.
> 
> How do I figure out what module (at least) this instantiated template
> is in?  Because of IFTI, I don't know what the template parameters
> are. Is there a way to figure this out from the call?

Use a debugger and step into it? ;-)


> I'm looking for a pragma(msg) or such that can give me the fully
> qualified name/instantiation details of this template.
[...]

AFAIK there isn't such a thing.


T

-- 
Живёшь только однажды.


Re: opBinary : Static ifs or specialization?

2020-06-23 Thread H. S. Teoh via Digitalmars-d-learn
On Tue, Jun 23, 2020 at 11:53:36PM +, claptrap via Digitalmars-d-learn 
wrote:
> So you have opBinary and half a dozen operators to implement. Do you
> use a separate method for each operator or do you have one method and
> a big static if else if to select code path?
[...]

If your implementations are based on built-in operators, you could use
mixins to unify the implementations into one, e.g.:

struct MyInt {
int i;
MyInt opBinary(string op)(MyInt arg) {
return MyInt(mixin("i "~op~" arg.i"));
}
}

which has far less boilerplate than separately implementing each
operator.  This is one of the main reasons for the design of .opBinary.


T

-- 
Let's eat some disquits while we format the biskettes.


Re: called copy constructor in foreach with ref on Range

2020-06-22 Thread H. S. Teoh via Digitalmars-d-learn
On Tue, Jun 23, 2020 at 03:25:55AM +, Stanislav Blinov via 
Digitalmars-d-learn wrote:
> On Tuesday, 23 June 2020 at 02:41:55 UTC, Jonathan M Davis wrote:
> 
> > As things stand, uncopyable ranges aren't really a thing, and common
> > range idiomns rely on ranges being copyable.
> 
> Which idioms are those? I mean, genuine idioms, not design flaws like
> e.g.  references.
> 
> > We'd need some major redesigning to make uncopyable ranges work, and
> > personally, I don't think that it's worth the trouble.
> 
> Of course we would. Andrei is entertaining changing the whole input
> range API. Though he, like you, seems opposed to the idea of
> uncopyables.
[...]

I'm also wondering what's the motivation behind supporting non-copyable
ranges, and whether it's worth the effort and inevitable complications
to support it if it's a rare use-case.  Do you have any compelling
use-case in mind, that might make this worthwhile?


T

-- 
Любишь кататься - люби и саночки возить. 


Re: called copy constructor in foreach with ref on Range

2020-06-22 Thread H. S. Teoh via Digitalmars-d-learn
On Mon, Jun 22, 2020 at 08:53:22PM -0600, Jonathan M Davis via 
Digitalmars-d-learn wrote:
[...]
> Exactly. It's because of issues like this that generic, range-based
> functions need to be tested with a variety of range types - including
> reference types. Without that, bugs are routinely missed.
[...]
> If we do actually rework the range API as has occasionally been
> discussed, it would be _really_ nice if we could more thoroughly
> restrict the copying semantics of ranges so that some of these
> problems go away, but without such a redesign, we're stuck with such
> problems.

Redesign or not, I think the new range API should be much more specific
in specifying exactly what behaviours ranges ought to have.  The current
API is under-specified, with two of the main problem points that I can
remember being:

1) Transient-ness: is the value return by .front guaranteed to remain
   valid, or does .popFront invalidate it?  This is not specified in the
   current spec, and we have ranges in Phobos of either type (e.g., iota
   and .byLine), with a lot of code just blindly assuming non-transience
   only to discover that things break down when handed a .byLine
   instance.

2) The exact semantics of copying (assigning) a range. What parts of a
   range's state is/isn't preserved when you assign a range to a local
   variable, for example?  What happens if you pass a range to a
   function and then continue to operate on the original?  All of this
   must be specified precisely in the range API so that we don't have
   one range acting one way and another range acting another way,
   thereby compromising the semantics of generic code.


T

-- 
Democracy: The triumph of popularity over principle. -- C.Bond


Re: called copy constructor in foreach with ref on Range

2020-06-22 Thread H. S. Teoh via Digitalmars-d-learn
On Mon, Jun 22, 2020 at 09:11:07PM +, Stanislav Blinov via 
Digitalmars-d-learn wrote:
> On Monday, 22 June 2020 at 20:51:37 UTC, Jonathan M Davis wrote:
[...]
> > And moving doesn't fix anything, since the original variable is
> > still there (just in its init state, which would still be invalid to
> > use in generic code and could outright crash in some cases if you
> > tried to use it - e.g. if it were a class reference, since it would
> > then be null).
> 
> Eh? A range in 'init' state should be an empty range. If you get a
> crash from that then there's a bug in that range's implementation, not
> in user code.

Don't be shocked when you find out how many Phobos ranges have .init
states that are invalid (e.g., non-empty, but .front and .popFront will
crash / return invalid values).


> > So, code that does a move could accidentally use the original range
> > after the move and have bugs just like code that copies the range
> > has bugs if the original is used after the copy has been made. So,
> > the rule of thumb is not that you should avoid copying ranges. It's
> > that once you've copied a range, you should then use only the copy
> > and not the original.
> 
> That is not true. Copying of forward ranges is absolutely fine. It's
> what the current `save()` primitive is supposed to do. It's the
> copying of input ranges should just be rejected, statically.

Jonathan is coming from the POV of generic code.  The problem with move
leaving the original range in its .init state isn't so much that it will
crash or anything (even though as you said that does indicate a flaw in
the range's implementation), but that the semantics of generic code
changes in subtle ways. For example:

auto myGenericFunc(R)(R r) {
...
foreach (x; r) {
doSomething(x);
}
if (!r.empty)
doSomethingElse(r);
...
}

Suppose for argument's sake that the above foreach/if structure is an
essential part of whatever algorithm myGenericFunc is implementing. Now
there's a problem, because if R has array-like semantics, then the
algorithm will do one thing, but if R has reference-like or move
semantics, then the behaviour of the algorithm will be different, even
if both ranges represent the same sequence of input values.

Note that in real-life code, this problem can be far more subtle than a
blatant foreach loop and if statement like the above. For example,
consider a function that drops the first n elements of a range. Your
generic function might want to pop the first n elements then do
something else with the rest of the range.  Well, if you write it the
obvious way:

auto myAlgo(R)(R r) {
size_t n = ...;
dropFirstN(r, n);
... // do something else with r
}

then you have a subtle bug, because the state of r after the call to
dropFirstN might be completely different depending on whether r behaves
like an array or like a by-reference or move type.


T

-- 
Truth, Sir, is a cow which will give [skeptics] no more milk, and so they are 
gone to milk the bull. -- Sam. Johnson


Re: how to skip the next (n) item & continue from (n+1) with a range ? e.g. in File(fn).byLine();

2020-06-22 Thread H. S. Teoh via Digitalmars-d-learn
On Mon, Jun 22, 2020 at 08:51:49PM +, mw via Digitalmars-d-learn wrote:
[...]
> >auto line = range.front;
> >range.popFront;  // pop immediately
[...]

This is dangerous, because it assumes .front is not invalidated by
.popFront.  It will not work, for example, with byLine because .front
returns a buffer which is reused by .popFront (a so-called "transient
range").

In the realm of defensive programming, it is better to make less
assumptions (don't assume .front remains valid after .popFront) than add
implicit assumptions (range is non-transient) that may break in subtle
ways that are not immediately obvious.


T

-- 
Why ask rhetorical questions? -- JC


Re: Parallel array append using std.parallelism?

2020-06-19 Thread H. S. Teoh via Digitalmars-d-learn
On Fri, Jun 19, 2020 at 06:48:18AM +, Simen Kjærås via Digitalmars-d-learn 
wrote:
[...]
> There's an example of exactly this in std.parallelism:
> https://dlang.org/phobos/std_parallelism.html#.TaskPool.workerIndex
> 
> In short:
> 
> Item[] targetArray = ...; // already contains data
> // Get thread count from taskPool
> Item[][] tmp = new Item[][taskPool.size+1];
> foreach (elem; input.parallel) {
> if (condition(elem)) {
> auto output = expensiveComputation(elem);
> // Use workerIndex as index
> tmp[taskPool.workerIndex] ~= output;
> }
> }
> foreach (a; tmp)
> targetArray ~= a;
[...]

Yes, that's exactly what I was looking for. Thanks!!


T

-- 
The best way to destroy a cause is to defend it poorly.


Parallel array append using std.parallelism?

2020-06-18 Thread H. S. Teoh via Digitalmars-d-learn
I have an array of input data that I'm looping over, and, based on some
condition, generate new items that are appended onto a target array
(which may already contain data). Since the creation of new items is
quite expensive, I'm thinking to parallelize it with parallel foreach.

To avoid data races, my thought is for each generated item to be
appended to thread-specific temporary arrays, that after the parallel
foreach get sequentially appended to the target array. Something like
this:

Item[] targetArray = ...; // already contains data
Item[][nThreads] tmp;
foreach (elem; input.parallel) {
if (condition(elem)) {
auto output = expensiveComputation(elem);
tmp[threadId] ~= output;
}
}
foreach (a; tmp)
targetArray ~= a;

Is there an easy way to achieve this with std.parallelism?  I looked
over the API but there doesn't seem to be any obvious way for a task to
know which thread it's running in, in order to know which tmp array it
should append to.  If possible I'd like to avoid having to manually
assign tasks to threads.


T

-- 
Questions are the beginning of intelligence, but the fear of God is the 
beginning of wisdom.


Re: Should a parser type be a struct or class?

2020-06-17 Thread H. S. Teoh via Digitalmars-d-learn
On Wed, Jun 17, 2020 at 02:32:09PM +, Adam D. Ruppe via Digitalmars-d-learn 
wrote:
> On Wednesday, 17 June 2020 at 14:24:01 UTC, Stefan Koch wrote:
> > Parser in dmd does even inherit from Lexer.
> 
> why would a parser ever inherit from a lexer?

Because, unlike a regular parser-driven compiler, dmd is a lexer-driven
one. :-D


T

-- 
The volume of a pizza of thickness a and radius z can be described by the 
following formula: pi zz a. -- Wouter Verhelst


Re: Should a parser type be a struct or class?

2020-06-17 Thread H. S. Teoh via Digitalmars-d-learn
On Wed, Jun 17, 2020 at 11:50:27AM +, Per Nordlöw via Digitalmars-d-learn 
wrote:
> Should a range-compliant aggregate type realizing a parser be encoded
> as a struct or class?

Preferably a struct IMO, but see below.


> In dmd `Lexer` and `Parser` are both classes.

Probably for historical reasons.


> In general how should I reason about whether an aggregate type should
> be encoded as a struct or class?

1) Does it need runtime polymorphism? If it does, use a class. If not,
probably a struct.

2) Does it make more sense as a by-value type, or a by-reference type?
In several of my projects, for example, I've had aggregate types start
out as structs (because of (1)), but eventually rewritten as (final)
classes because I started finding myself using `ref` or `&` everywhere
to get by-reference semantics.

My rule-of-thumb is basically adopted from TDPL: a struct as a
"glorified int" with by-value semantics, a class is a more traditional
OO object. If my aggregate behaves like a glorified int, then a struct
is a good choice. If it behaves more like a traditional OO encapsulated
type, then a class is probably the right answer.


T

-- 
Many open minds should be closed for repairs. -- K5 user


Re: final switch problem

2020-06-13 Thread H. S. Teoh via Digitalmars-d-learn
On Sat, Jun 13, 2020 at 09:02:21AM +, John Chapman via Digitalmars-d-learn 
wrote:
[...]
> module test;
> 
> import std.uni;
> 
> enum Cheese { cheddar, edam }
> 
> void test(Cheese cheese) {
>   final switch (cheese) {
>   case Cheese.cheddar: break;
>   case Cheese.edam: break;
>   }
> }
> 
> void main() {
>   test(Cheese.cheddar);
> }
> ---
> 
> error LNK2019: unresolved external symbol "pure nothrow @nogc @safe void
> core.internal.switch_.__switch_error!().__switch_error(immutable(char)[],
> ulong)" (_D4core8internal7switch___T14__switch_errorZQrFNaNbNiNfAyamZv)
> referenced in function _Dmain
[...]

Tested it on git master, DMD 2.091.1-beta.1 (Linux/64), could not
reproduce problem.

>From the error message, my first guess would be that your compiler and
druntime are out-of-sync somehow.


T

-- 
My father told me I wasn't at all afraid of hard work. I could lie down right 
next to it and go to sleep. -- Walter Bright


Re: Looking for a Code Review of a Bioinformatics POC

2020-06-12 Thread H. S. Teoh via Digitalmars-d-learn
On Fri, Jun 12, 2020 at 12:11:44PM +, duck_tape via Digitalmars-d-learn 
wrote:
> On Friday, 12 June 2020 at 12:02:19 UTC, duck_tape wrote:
> > For speedups with getting my hands dirty:
> > - Does writef and company flush on every line? I still haven't found
> > the source of this.

writef, et al, ultimately goes through LockingTextWriter in
std.stdio.File:

https://github.com/dlang/phobos/blob/master/std/stdio.d#L2890

Looks like it's doing some Unicode manipulation and writing character by
character -- a pretty slow proposition IMO!  It was done this way for
Unicode-correctness, AFAICT, but if you already know the final form your
output is going to take, directly calling fwrite(), or the D wrapper
File.rawWrite(), will probably give you a significant performance boost.


> > - It looks like I could use {f}printf if I really wanted to:
> > https://forum.dlang.org/post/hzcjbanvkxgohkbvj...@forum.dlang.org

Be aware that D strings, other than string literals, are generally NOT
null-terminated, so you need to call toStringZ before calling fprintf,
otherwise you might be in for a nasty surprise. :-P  Other than that,
calling C from D is pretty easy:

extern(C) int printf(char*, ...);

void myDCode(string data) {
printf("%s\n", data.toStringZ); // calls C printf
}


> On Friday, 12 June 2020 at 12:02:19 UTC, duck_tape wrote:
> 
> Switching to using `core.stdc.stdio.printf` shaved off nearly two
> seconds (11->9)!
> 
> Once I wrap this up for submission to biofast I will play with mem
> memmapping / iopipe / tsvutils buffered writers. Sambamba is also
> doing some non-standard tweaks to it's outputting as well.
> 
> I'm still convinced that stdout is flushing by line.

It seems likely, if you're outputting to terminal. Otherwise, it's
likely the performance slowdown is caused by Unicode manipulation code
inside LockingTextWriter.

On that note, somebody should get to the bottom of this, and submit a PR
to Phobos with a fast-track path for the (IMO very common) case where
the string can just be fwrite'd straight into output. AFAICT, all the
extra baggage currently in LockingTextWriter is mainly to deal with the
case where the OS expects a (slightly) different encoding for text than
is internally represented, e.g., classic 0x0D 0x0A DOS line endings
(which I heard are obsolete these days, so even that case may not be as
common as it used to be anymore), or outputting UTF-16 to UTF-8 or vice
versa.  I'm skeptical whether this is the common case these days, so
having a fast path for UTF-8 -> UTF-8 (i.e., just fwrite the whole
thing straight to file) will be a good improvement for D.


T

-- 
Nobody is perfect.  I am Nobody. -- pepoluan, GKC forum


Re: What is the current stage of @property ?

2020-06-12 Thread H. S. Teoh via Digitalmars-d-learn
On Thu, Jun 11, 2020 at 10:46:32AM +, Paul Backus via Digitalmars-d-learn 
wrote:
> On Thursday, 11 June 2020 at 05:41:25 UTC, H. S. Teoh wrote:
> > 
> > Ironically, just today I ran into this corner case where @property
> > actually became a solution to a real problem:
> > 
> > //-module1.d--
> > auto someGenericFunc(T)(T t) {
> > ...
> > static if (is(typeof(T.init.method) : T)) {
> > R someRange = ...; // ElementType!R == T
> > auto value = someRange.map!(e => e.method);
> > }
> > ...
> > }
> 
> Personally I like the way Phobos does it:
> 
> ReturnType!((T t) => t.method)
> 
> This works for both @property and non-@property methods.

Mmm, very nice!  I'll keep this in mind for next time!


T

-- 
The two rules of success: 1. Don't tell everything you know. -- YHL


Re: Finding the file and unittest that triggers an ICE during dub project build only when unittests are enabled

2020-06-12 Thread H. S. Teoh via Digitalmars-d-learn
On Fri, Jun 12, 2020 at 09:29:06PM +, MoonlightSentinel via 
Digitalmars-d-learn wrote:
> On Friday, 12 June 2020 at 18:18:25 UTC, Per Nordlöw wrote:
> > How do I most easily track down which unittest in which file that
> > causes the crash?
[...]

Compile your program with debugging symbols, and then run it in gdb.
When it crashes, the `bt` command should show you the stack trace. Near
the bottom of the trace (top of the stack) should be the unittest
function with a name that looks like _D4test16__unittest_L5_C1FZv, where
L5 means the unittest that started on line 1 of the source, and C1 means
the column on that line, and "FZv" is the mangling of the unittest's
attributes, and "_D4test" is the mangled name of the module,
corresponding to "test".


T

-- 
The trouble with TCP jokes is that it's like hearing the same joke over and 
over.


Re: Looking for a Code Review of a Bioinformatics POC

2020-06-12 Thread H. S. Teoh via Digitalmars-d-learn
On Fri, Jun 12, 2020 at 03:32:48AM +, Jon Degenhardt via 
Digitalmars-d-learn wrote:
[...]
> I haven't spent much time on results presentation, I know it's not
> that easy to read and interpret the results. Brief summary - On files
> with short lines buffering will result in dramatic throughput
> improvements over the standard phobos facilities. This is true for
> both input and output, through likely for different reasons. For input
> iopipe is the fastest available. tsv-utils buffered facilities are
> materially faster than phobos for both input and output, but not as
> fast as iopipe for input. Combining iopipe for input with tsv-utils
> BufferOutputRange for output works pretty well.
> 
> For files with long lines both iopipe and tsv-utils BufferedByLine are
> materially faster than Phobos File.byLine when reading. For writing
> there wasn't much difference from Phobos File.write.

Interesting.  Based on the OP's posted profile data, I got the
impression that input wasn't a big deal, but output was.  I wonder why.


> A note on File.byLine - I've had many opportunities to compare Phobos
> File.byLine to facilities in other programming languages, and it is
> not bad at all. But it is beatable.

I glanced over the implementation of byLine.  It appears to be the
unhappy compromise of trying to be 100% correct, cover all possible UTF
encodings, and all possible types of input streams (on-disk file vs.
interactive console).  It does UTF decoding and resizing of arrays, and
a lot of other frilly little squirrelly things.  In fact I'm dismayed at
how hairy it is, considering the conceptual simplicity of the task!

Given this, it will definitely be much faster to load in large chunks of
the file at a time into a buffer, and scanning in-memory for linebreaks.
I wouldn't bother with decoding at all; I'd just precompute the byte
sequence of the linebreaks for whatever encoding the file is expected to
be in, and just scan for that byte pattern and return slices to the
data.


> About Memory Mapped Files - The benchmarks don't include compare
> against mmfile. They certainly make sense as a comparison point.
[...]

I'd definitely seriously consider using std.mmfile if I/O is determined
to be a significant bottleneck. Letting the OS page in the file
on-demand for you instead of copying buffers across the C file API
boundary is definitely going to be a lot faster.  Plus it will greatly
simplify the code -- you could just arbitrarily scan and slice over file
data without needing to manually manage buffers on your own, so your
code will be much simpler and conducive for the compiler to squeeze the
last bit of speed juice out of.

I'd definitely avoid stdio.byLine if input was determined to be a
bottleneck: decoding characters from file data just to find linebreaks
seems to me to be definitely a slow way of doing things.

Having said all of that, though: usually in non-trivial programs reading
input is the least of your worries, so this kind of micro-optimization
is probably unwarranted except for very niche cases and for
micro-benchmarks and other such toy programs where the cost of I/O
constitutes a significant chunk of running times.  But knowing what
byLine does under the hood is definitely interesting information for me
to keep in mind, the next time I write an input-heavy program.

(I'm reminded of that one time when, as little diversion, I decided to
see if I could beat GNU wc at counting lines in a file. It was not easy
to beat since wc it's optimized to next year and back, but eventually a
combination of std.mmfile and std.parallelism to scan large chunks of
file simultaneously managed to beat wc by a good margin.  In the
meantime, though, I also discovered that a file of very short lines
triggers poor performance out of wc, whereas a file of very long lines
triggers the best performance -- because glibc's memchr appeared to be
optimized for a micro-benchmark geared towards scanning arrays with only
rare occurrences of the sought character, but typical text files exhibit
much more frequent matches (shorter lines). When fed a file of very
short lines, the overhead of the hyper-optimized code added up
significantly in the latter case, whereas when lines were sufficiently
long it far outweighed the overhead cost. Optimization is a tricky
beast: always make sure to measure and optimize for your actual use case
rather than making your code look good on some artificial
micro-benchmark, else your code may look good on the benchmark but
actually perform poorly on real-world data.)


T

-- 
May you live all the days of your life. -- Jonathan Swift


Re: Looking for a Code Review of a Bioinformatics POC

2020-06-11 Thread H. S. Teoh via Digitalmars-d-learn
On Thu, Jun 11, 2020 at 11:02:21PM +, duck_tape via Digitalmars-d-learn 
wrote:
[...]
> I will give that a shot! Also of interest, the profiler results on a
> full runthrough do show file writing and int parsing as the 2nd and
> 3rd most time consuming activities:
> 
> ```
>   Num  TreeFuncPer
>   CallsTimeTimeCall
> 
> 8942869   46473   44660   0 void app.IITree!(int, 
> bool).IITree.overlap(int, int, void delegate(app.IITree!(int, 
> bool).IITree.Interval))
> 8942869   330659656   0 @safe void 
> std.stdio.File.write!(char[], immutable(char)[], char[], immutable(char)[], 
> char[], immutable(char)[], int, immutable(char)[], int, char).write(char[], 
> immutable(char)[], char[], immutable(char)[], char[], immutable(char)[], int, 
> immutable(char)[], int, char)
> 20273052   100249569   0 pure @safe int 
> std.conv.parse!(int, char[]).parse(ref char[])

Hmm, looks like it's not so much input that's slow, but *output*. In
fact, it looks pretty bad, taking almost as much time as overlap() does
in total!

This makes me think that writing your own output buffer could be
worthwhile.  Here's a quick-n-dirty way of doing that:

import std.array : appender;
auto filebuf = appender!string;
...
// Replace every call to writeln with this:
put(filebuf, text(... /* arguments go here */ ..., "\n"));

...

// At the end of the main loop:
enum bufLimit = 0x1000; // whatever the limit you want
if (filebuf.length > someLimit) {
write(filebuf.data); // flush output data
stdout.flush;
filebuf.clear;
}

This is just a rough sketch for an initial test, of course.  For a truly
optimized output buffer I'd write a container struct with methods for
managing the appending and flushing of output. But this is just to get
an idea of whether it actually improves performance before investing
more effort into going in this direction.


T

-- 
He who sacrifices functionality for ease of use, loses both and deserves 
neither. -- Slashdotter


Re: Looking for a Code Review of a Bioinformatics POC

2020-06-11 Thread H. S. Teoh via Digitalmars-d-learn
On Thu, Jun 11, 2020 at 10:41:12PM +, duck_tape via Digitalmars-d-learn 
wrote:
> On Thursday, 11 June 2020 at 22:19:27 UTC, H. S. Teoh wrote:
> > To encourage inlining, you could make it an alias parameter instead
> > of a delegate, something like this:
> > 
> > void overlap(alias cb)(SType start, SType stop) { ... }
> > ...
> > bed[chr].overlap!callback(st0, en0);
> > 
> 
> 
> I don't think ldc can handl that yet. I get an error saying
> 
> ```
> source/app.d(72,7): Error: function app.main.overlap!(callback).overlap
> requires a dual-context, which is not yet supported by LDC
> ```
> 
> And I see an open ticket for it on the ldc project.

Oh right. :-(  But in any case, I'm a little skeptical whether this is
the performance bottleneck anyway.

But one simple thing to try is to add 'scope' to the callback parameter,
which could potentially save you a GC allocation. I'm not 100% certain
this will make a difference, but since it's such an easy change it's
worth a shot.


T

-- 
Philosophy: how to make a career out of daydreaming.


Re: Looking for a Code Review of a Bioinformatics POC

2020-06-11 Thread H. S. Teoh via Digitalmars-d-learn
On Thu, Jun 11, 2020 at 04:13:34PM +, duck_tape via Digitalmars-d-learn 
wrote:
[...]
> Currently my D version is a few seconds slower than the Crystal
> version.  putting it very solid in third place overall. I'm not really
> sure where it's falling behind crystal since `-release` removes bounds
> checking. I have not looked at the assembly between the two, but I
> suspect that Crystal inlines the callback and D does not.

To encourage inlining, you could make it an alias parameter instead of a
delegate, something like this:

void overlap(alias cb)(SType start, SType stop) { ... }
...
bed[chr].overlap!callback(st0, en0);

This doesn't guarantee inlining, though. And no guarantee it will
actually improve performance.


> I also think there is room for improvement in the IO, as I'm just
> using the defaults.

I wouldn't spend too much time optimizing I/O without profiling it
first, to check that it's actually a bottleneck. If I/O turns out to be
a real bottleneck, you could try using std.mmfile.MmFile to mmap the
input directly into the program's address space, which should give you a
speed boost.


T

-- 
An imaginary friend squared is a real enemy.


Re: What is the current stage of @property ?

2020-06-10 Thread H. S. Teoh via Digitalmars-d-learn
On Wed, Jun 10, 2020 at 10:58:57PM -0600, Jonathan M Davis via 
Digitalmars-d-learn wrote:
[...]
> As things stand, @property has no real practical purpose but
> frequently gets used to indicate that it's the intention of a
> function's author for it to be used as if it were a variable. I
> suspect that it's also frequently misunderstand that it's required if
> you want to call a function without parens. So, you're likely to see
> @property in quite a lot of D code, but ultimately, all it's really
> doing is serving as documentation of the author's intent and screwing
> up metaprogramming.
[...]

Ironically, just today I ran into this corner case where @property
actually became a solution to a real problem:

//-module1.d--
auto someGenericFunc(T)(T t) {
...
static if (is(typeof(T.init.method) : T)) {
R someRange = ...; // ElementType!R == T
auto value = someRange.map!(e => e.method);
}
...
}

//-module2.d--
struct MyType {
...
auto method()() {
return ...;
}
...
}

//--module3.d-
import module1;
import module2;
auto someFunc(...) {
auto result = someGenericFunc!MyType;
}

This was failing compilation, because when MyType.method is a template,
the static if condition in module1 fails.  Adding @property to
MyType.method tells the compiler that `T.init.method` is intended to
refer to the return value rather than the template function itself, and
thus neatly solves the problem.


T

-- 
It's amazing how careful choice of punctuation can leave you hanging:


Re: What is the current stage of @property ?

2020-06-10 Thread H. S. Teoh via Digitalmars-d-learn
On Wed, Jun 10, 2020 at 10:50:17PM +, Paul Backus via Digitalmars-d-learn 
wrote:
> On Wednesday, 10 June 2020 at 21:41:54 UTC, H. S. Teoh wrote:
> > There are a few places where it's needed (like satisfying the range
> > API, which implicitly checks for it)
> 
> That may have been true at one point, but it isn't true now:
> 
> struct S {
> bool empty() { return false; }
> int front() { return 0; }
> void popFront() {}
> }
> 
> static assert(isInputRange!S); // passes

Nice.  So pretty soon I can get rid of @property spam from my code,
which is really not very useful and only adds extra keystrokes for
little benefit.


T

-- 
Debugging is twice as hard as writing the code in the first place. Therefore, 
if you write the code as cleverly as possible, you are, by definition, not 
smart enough to debug it. -- Brian W. Kernighan


Re: What is the current stage of @property ?

2020-06-10 Thread H. S. Teoh via Digitalmars-d-learn
On Wed, Jun 10, 2020 at 08:24:19PM +, Vinod K Chandran via 
Digitalmars-d-learn wrote:
> Hi all,
> I read in an old thread that authors of D wants to eliminate
> @property. I just roughly read the big thread bu couldn't find a
> conclusion. After all that thread is a 48 page longer jumbo thread. So
> out of curiosity, i am asking this. What is the current state of
> @property ? Is it deprecated ?

It's stuck in limbo, like many things that people just cannot agree on.
There are a few places where it's needed (like satisfying the range API,
which implicitly checks for it), but for the most part, you can just
ignore it, it doesn't really make a big difference.  Life goes on.


T

-- 
English has the lovely word "defenestrate", meaning "to execute by throwing 
someone out a window", or more recently "to remove Windows from a computer and 
replace it with something useful". :-) -- John Cowan


Re: Why is there no range iteration with index by the language?

2020-06-09 Thread H. S. Teoh via Digitalmars-d-learn
On Tue, Jun 09, 2020 at 05:03:55PM -0700, H. S. Teoh via Digitalmars-d-learn 
wrote:
> On Tue, Jun 09, 2020 at 11:53:16PM +, Q. Schroll via Digitalmars-d-learn 
> wrote:
> > Is there any particular reason why std.range : enumerate is a thing
> > and
> > 
> > foreach (i, e; range) { ... }
> > 
> > doesn't work from the get-go?
> [...]
> 
> std.range.indexed is your friend. ;-)
[...]

Aaah, that function doesn't do what I thought it did.  Sorry!! :-(

What you want is std.range.enumerate.  But you already knew that.


T

-- 
EMACS = Extremely Massive And Cumbersome System


Re: Why is there no range iteration with index by the language?

2020-06-09 Thread H. S. Teoh via Digitalmars-d-learn
On Tue, Jun 09, 2020 at 11:53:16PM +, Q. Schroll via Digitalmars-d-learn 
wrote:
> Is there any particular reason why std.range : enumerate is a thing
> and
> 
> foreach (i, e; range) { ... }
> 
> doesn't work from the get-go?
[...]

std.range.indexed is your friend. ;-)


T

-- 
Let's eat some disquits while we format the biskettes.


Re: Is there a list of things which are slow to compile?

2020-06-05 Thread H. S. Teoh via Digitalmars-d-learn
On Fri, Jun 05, 2020 at 08:25:13AM +, aberba via Digitalmars-d-learn wrote:
> On Wednesday, 3 June 2020 at 17:02:35 UTC, H. S. Teoh wrote:
> > On Wed, Jun 03, 2020 at 09:36:52AM +, drathier via
> > Digitalmars-d-learn wrote:
> > > I'm wondering if there's a place that lists things which are
> > > slower/faster to compile? DMD is pretty famed for compiling
> > > quickly, but I'm not seeing particularly high speed at all, and I
> > > want to fix that.
> > 
> > The two usual culprits are:
> > - Recursive/chained templates
> > - Excessive CTFE
[...]
> I'm thinking about a resource hub for D with information like these.
> Can I use this information?
[...]

Of course. No need to reference this thread, what I wrote above is
pretty much common knowledge for anyone who has worked with D long
enough.


T

-- 
Famous last words: I *think* this will work...


Re: how to achieve C's Token Pasting (##) Operator to generate variable name in D?

2020-06-04 Thread H. S. Teoh via Digitalmars-d-learn
On Thu, Jun 04, 2020 at 06:09:35PM +, mw via Digitalmars-d-learn wrote:
[...]
> --
> class Person : NameI, AddrI {
>   mixin NameT!Person rename equals as name_equals;
>   mixin AddrT!Person rename equals as addr_equals;
> 
>   bool equals(Person other) {
> return this.name_equals(other) &&
>this.addr_equlas(other);
>   }
> }
> --

TBH, whenever I run into a diamond inheritance problem or similar, my
first reaction is, I'm using the wrong tool for modelling my data; I
should be using some kind of component-based system instead of OO
inheritance.

Nowadays I rarely use OO-style inheritance for data modelling anymore;
it's still useful for rare cases where a straight hierarchy makes sense
(traditional GUI widgets, for example, or parse trees), but for complex
modelling I just stop pretending that there's a direct mapping from
problem domain to language constructs, and instead build containers that
have arbitrary component combinations as an infrastructure instead.

Recently I've been dabbling in ECS (entity-component-system) adaptations
from gamedev: the system part is not useful to me, but the idea behind
entity-component storage is very useful for modelling complex data, much
more suitable than traditional OO inheritance IMO.


T

-- 
I am Ohm of Borg. Resistance is voltage over current.


Re: writeln Function while reading a Text File is printing appending text "before text" and "after text" at the same position

2020-06-03 Thread H. S. Teoh via Digitalmars-d-learn
On Wed, Jun 03, 2020 at 08:43:43PM +, BoQsc via Digitalmars-d-learn wrote:
> On Wednesday, 3 June 2020 at 20:05:52 UTC, ttk wrote:
[...]
> > That works, but consider using chomp() instead.
> > 
> > https://dlang.org/phobos/std_string.html#.chomp
> 
> Chomp sounds kind of funny hahaha.
> Who came up with these function names? Walter Bright?
> Anyways, Chomp's way looks way more simple. Thanks.

Chomp comes from Perl.


T

-- 
For every argument for something, there is always an equal and opposite 
argument against it.
Debates don't give answers, only wounded or inflated egos.


Re: Is there a list of things which are slow to compile?

2020-06-03 Thread H. S. Teoh via Digitalmars-d-learn
On Wed, Jun 03, 2020 at 09:36:52AM +, drathier via Digitalmars-d-learn 
wrote:
> I'm wondering if there's a place that lists things which are
> slower/faster to compile? DMD is pretty famed for compiling quickly,
> but I'm not seeing particularly high speed at all, and I want to fix
> that.

The two usual culprits are:
- Recursive/chained templates
- Excessive CTFE

Note that while the current CTFE engine is slow, it's still reasonably
fast for short computations. Just don't write nested loops or loops with
a huge number of iterations inside your CTFE code, and you should be
fine. And on that note, even running std.format with all of its
complexity inside CTFE is reasonably fast, as long as you don't do it
too often; so generally you won't see a problem here unless you have
loop with too many iterations or too deeply-nested loops running in
CTFE.

Templates are generally reasonably OK, until you use too many recursive
templates. Or if you chain too many of them together, like if you have
excessively long UFCS chains with Phobos algorithms. Short chains are
generally OK, but once they start getting long they will generate large
symbols and large numbers of instantiations. Large symbols used to be a
big problem, but ever since Rainer's fix they have generally been a lot
tamer. But still, it's something to avoid unless you can't help it.

Recursive templates are generally bad because they tend to produce a
super-linear number of instantiations, which consume lots of compiler
memory and also slow things down. Use too many of them, and things will
quickly slow to a crawl.

Worst is if you combine both deeply-nested templates and CTFE, like
std.regex does. Similarly, std.format (which includes writefln & co)
tends to add 1-2 seconds to compile time.

Another is if you have an excessively long function body, IIRC there are
some O(n^2) algorithms in the compiler w.r.t. the length of the function
body. But I don't expect normal code to reach the point where this
begins to matter; generally you won't run into this unless your code is
*really* poorly written (like the entire application inside main()), or
you're using excessive code generation (like the mixin of a huge
procedurally generated string).

Identifier lengths are generally no problem unless you're talking about
100KB-long identifiers, which used to be a problem until Rainer
implemented backreferences in the mangling. But I don't expect normal
code to generate symbols of this order of magnitude unless you're using
excessively-long UFCS chains with nested templates. Identifier length
generally doesn't even register on the radar unless they're ridiculously
long, like tens or hundreds of KB long -- not something a human would
type. What humans would consider a long identifier, like Java-style
names that span 50 characters, are mere round-off error and probably
don't even make a measurable difference. The problem really only begins
to surface when you have 10,000 characters in your identifier or larger.

Comments are not even a blip on the radar: lexing is the fastest part of
the compilation process.  Similarly, aliases are extremely cheap, it's
not even on the radar. Delegates have only a runtime cost; they are
similarly unnoticeably cheap during compilation.  As are Variants,
unless you're running Variants inside CTFE (which I don't think even
works).


T

-- 
Why waste time reinventing the wheel, when you could be reinventing the engine? 
-- Damian Conway


Re: Postblit segfault.

2020-06-01 Thread H. S. Teoh via Digitalmars-d-learn
On Mon, Jun 01, 2020 at 03:27:11PM +, Basile B. via Digitalmars-d-learn 
wrote:
[...]
> Possibly a backend bug (keyword "wrong code"), caused by either of [1] or
> [2]
> 
> [1] https://github.com/dlang/dmd/pull/9357
> [2] https://github.com/dlang/dmd/pull/9623/files

Yeah, it looks like a backend bug.  The struct address was somehow
either not loaded correctly, or not passed correctly (didn't look into
more detail which case).

A regression should be filed for this, if not already.


T

-- 
It's amazing how careful choice of punctuation can leave you hanging:


  1   2   3   4   5   6   7   8   9   10   >