Re: how to assign to shared obj.systime?

2020-07-10 Thread Jonathan M Davis via Digitalmars-d-learn
On Friday, July 10, 2020 12:30:16 PM MDT mw via Digitalmars-d-learn wrote:
> On Friday, 10 July 2020 at 17:35:56 UTC, Steven Schveighoffer
>
> wrote:
> > Mark your setTime as shared, then cast away shared (as you
> > don't need atomics once it's locked), and assign:
> >
> > synchronized setTime(ref SysTime t) shared {
> >
> > (cast()this).time = t;
> >
> > }
>
> I know I can make it work by casting, my question is:
>
> we had a lock on the owning shared object already, WHY we still
> need the cast to make it compile.

Because the type system has no way of knowing that access to that shared
object is currently protected, and baking that into the type system is
actually very difficult - especially if you don't want to be super
restrictive about what is allowed.

The only scheme that anyone has come up thus far with which would work is
TDPL's synchronized classes (which have never been implemented), but in
order for them to work, they would have to be restrictive about what you do
with the member variables, and ultimately, the compiler would still only be
able to implicitly remove the outer layer of shared (i.e. the layer sitting
directly in the class object itself), since that's the only layer that the
compiler could prove hadn't had any references to it escape. So, you'd have
to create a class just to be able to avoid casting, and it wouldn't
implicitly remove enough of shared to be useful in anything but simple
cases.

Sure, it would be great if we could have shared be implicitly removed when
the object in question is protected by a mutex, but the type system would
have to know that that mutex was associated with that object and be able to
prove not only that that mutex was locked but that no other piece of code
could possibly access that shared object without locking that mutex. It
would also have to be able to prove that no thread-local references escaped
from the code where shared was implicitly removed. It's incredibly difficult
to bake the required information into the type system even while be very
restrictive about what's allowed let alone while allowing code to be as
flexible as code generally needs to be - especially in a systems language
like D.

If someone actually manages to come up with an appropriate scheme that lets
us implicitly removed shared under some set of circumstances, then we may
very well get that ability at some point in the future, but it seems very
unlikely as things stand, and even if someone did manage it, it's even less
likely that it would work outside of a limited set of use cases, since there
are a variety of ways of dealing with safely accessing data across threads.

So, for the forseeable future, explicit casts are generally going to be
required when dealing with shared.

- Jonathan M Davis





Re: how to assign to shared obj.systime?

2020-07-10 Thread Jonathan M Davis via Digitalmars-d-learn
On Thursday, July 9, 2020 9:01:20 PM MDT mw via Digitalmars-d-learn wrote:
> On Friday, 10 July 2020 at 02:59:56 UTC, mw wrote:
> > Error: template std.datetime.systime.SysTime.opAssign cannot
> > deduce function from argument types !()(SysTime) shared,
> > candidates are:
> > /usr/include/dmd/phobos/std/datetime/systime.d(659,17):
> > opAssign()(auto ref const(SysTime) rhs)
>
> of course, better without casting.

Unless you're dealing with a primitive type that works with atomics, you
pretty much always have to cast when using shared (the only real exception
being objects that are specifically designed to work as shared and do the
atomics or casting internally for you). In general, when operating on a
shared object, you need to protect the section of code that's operating on
it with a mutex and then temporarily cast away shared to operate on the
object as thread-local. It's then up to you to ensure that no thread-local
references to the shared data escape the section of code protected by the
mutex (though scope may help with that if used in conjunction with
-dip1000).

- Jonathan M Davis





Re: What's the point of static arrays ?

2020-07-09 Thread Jonathan M Davis via Digitalmars-d-learn
On Thursday, July 9, 2020 10:21:41 AM MDT H. S. Teoh via Digitalmars-d-learn 
wrote:
> > - Assignment copies the whole array, as in int[5] a; auto b = a;
>
> Sometimes this is desirable.  Consider the 3D game example.  Suppose
> you're given a vector and need to perform some computation on it. If it
> were a dynamic array, you'd need to allocate a new array on the heap in
> order to work on it without changing the original vector. With a static
> array, it's passed by value to your function, so you just do what you
> need to do with it, and when you're done, either discard it (== no work
> because it's allocated on the stack) or return it (== return on the
> stack, no allocations).

I recall that at one point, I wrote brute-force sudoku solver, and
initially, I'd used dynamic arrays to represent the board. When I switched
them to static arrays, it was _way_ faster - presumably, because all of
those heap allocations were gone. And of course, since the sudoku board is
always the same size, the ability to resize the array was unnecessary.

In most programs that I've written, it hasn't made sense to use static
arrays anywhere, but sometimes, they're exactly what you need.

- Jonathan M Davis





Re: called copy constructor in foreach with ref on Range

2020-06-22 Thread Jonathan M Davis via Digitalmars-d-learn
On Monday, June 22, 2020 9:25:55 PM MDT Stanislav Blinov via Digitalmars-d-
learn wrote:
> On Tuesday, 23 June 2020 at 02:41:55 UTC, Jonathan M Davis wrote:
> > As things stand, uncopyable ranges aren't really a thing, and
> > common range idiomns rely on ranges being copyable.
>
> Which idioms are those? I mean, genuine idioms, not design flaws
> like e.g. references.

It is extremely common to wrap ranges in other ranges (and in fact, you
basically have to in order to have lazy ranges). That really doesn't work
very well - if at all - if you can't copy the range. It might be possible
with a bunch of explicit calls to move, but that would result in range-based
code in general being @system without a clean way to use @trusted, since
whether it's really safe to mark such moves with @trusted would depend on
the specific range implementation, which then is a big problem with generic
code. Regardless of whether it's actually possible to make it work though,
it's a complete mess in comparison to simply copying. And the fact that
chaining range-based calls is extremely common makes the problem that much
worse.

The way that ranges are routinely passed around and wrapped works as well as
it does, because ranges are copyable.

> > As things stand, it is _not_ true that it's safe to copy
> > forward ranges and then use the original. Sure, it will work
> > for some ranges, but for others it won't. The entire reason
> > that save exists is for ranges that are classes, because
> > copying them does not result in an independent range. The range
> > API does not require that copying a range result in an
> > independent copy. It's not even guaranteed that copying a range
> > that's a struct will result in an indpendent copy. Depending on
> > the range type, copying it could result in an independent copy,
> > or it could result in a reference to to the original range, or
> > it could even result in part of the state being copied and part
> > of it being a reference (e.g. if the current front is stored
> > directly in the range, but the iteration state is stored by
> > reference).
>
> If copying isn't making a copy, it's not copying.

The semantics of copying a variable or object vary wildly depending on the
type regardless of whether we're talking about ranges. Copying a pointer or
reference is still a copy even if it isn't a deep copy. Copying a range
_always_ results in a copy, just like it does with any other type. It just
doesn't necessarily result in a copy which can be independently iterated.

- Jonathan M Davis





Re: called copy constructor in foreach with ref on Range

2020-06-22 Thread Jonathan M Davis via Digitalmars-d-learn
On Monday, June 22, 2020 3:33:08 PM MDT H. S. Teoh via Digitalmars-d-learn 
wrote:
> On Mon, Jun 22, 2020 at 09:11:07PM +, Stanislav Blinov via Digitalmars-
d-learn wrote:
> > That is not true. Copying of forward ranges is absolutely fine. It's
> > what the current `save()` primitive is supposed to do. It's the
> > copying of input ranges should just be rejected, statically.
>
> Jonathan is coming from the POV of generic code.  The problem with move
> leaving the original range in its .init state isn't so much that it will
> crash or anything (even though as you said that does indicate a flaw in
> the range's implementation), but that the semantics of generic code
> changes in subtle ways. For example:
>
>   auto myGenericFunc(R)(R r) {
>   ...
>   foreach (x; r) {
>   doSomething(x);
>   }
>   if (!r.empty)
>   doSomethingElse(r);
>   ...
>   }
>
> Suppose for argument's sake that the above foreach/if structure is an
> essential part of whatever algorithm myGenericFunc is implementing. Now
> there's a problem, because if R has array-like semantics, then the
> algorithm will do one thing, but if R has reference-like or move
> semantics, then the behaviour of the algorithm will be different, even
> if both ranges represent the same sequence of input values.
>
> Note that in real-life code, this problem can be far more subtle than a
> blatant foreach loop and if statement like the above. For example,
> consider a function that drops the first n elements of a range. Your
> generic function might want to pop the first n elements then do
> something else with the rest of the range.  Well, if you write it the
> obvious way:
>
>   auto myAlgo(R)(R r) {
>   size_t n = ...;
>   dropFirstN(r, n);
>   ... // do something else with r
>   }
>
> then you have a subtle bug, because the state of r after the call to
> dropFirstN might be completely different depending on whether r behaves
> like an array or like a by-reference or move type.

Exactly. It's because of issues like this that generic, range-based
functions need to be tested with a variety of range types - including
reference types. Without that, bugs are routinely missed. I think that a
fairly large percentage of Phobos (maybe even all of it) gets that right
now, because we've added tests for reference type ranges, but there used to
be quite a few bugs in Phobos, because there were no such tests. It's
frequently the case that people write range-based code under the assumption
that ranges will act like dynamic arrays, and their code then also works
with many ranges which have similar copying behavior, but it doesn't work
with all range types. Such problems don't generally get caught without
extensive testing. It matters a lot less for code within a program that
doesn't use a large variety of range types, but for libraries using generic
functions, it's a must.

If we do actually rework the range API as has occasionally been discussed,
it would be _really_ nice if we could more thoroughly restrict the copying
semantics of ranges so that some of these problems go away, but without such
a redesign, we're stuck with such problems. And even with a redesign, the
best fix for it is far from obvious, because basic input ranges and forward
ranges inherently have different copying semantics. We can probably require
that copying forward ranges always results in an independent copy (thereby
essentially having value semantics), but basic input ranges can't really be
value types, because if they could, they could be forward ranges, meaning
that they're generally going to either be reference types or
pseudo-reference types, which of course have different copying semantics.
So, as long as the range API is set up so that the same code can operate on
both basic input and forward ranges, we pretty much inherently have a
problem with the copying semantics in ranges being inconsistent - though
requiring value semantics for forward ranges would still be a significant
improvement.

- Jonathan M Davis





Re: called copy constructor in foreach with ref on Range

2020-06-22 Thread Jonathan M Davis via Digitalmars-d-learn
On Monday, June 22, 2020 3:11:07 PM MDT Stanislav Blinov via Digitalmars-d-
learn wrote:
> On Monday, 22 June 2020 at 20:51:37 UTC, Jonathan M Davis wrote:
> > You're unlikely to find much range-based code that does that
> > and there really isn't much point in doing that. Again, copying
> > isn't the problem. It's using the original after making the
> > copy that's the problem.
>
> Copy *is* the problem. If you can't make a copy (i.e. you get a
> compilation error) - you don't have a problem, the compiler just
> found a bug for you. Sadly, historically D's libraries were built
> around lots of redundant copying, even though there are very few
> cases where copies are actually required. The reason why we're
> "unlikely to find much range-based code that does that [move]" is
> (a) sloppy or shortsighted design and (b) poor language support.
> The latter hopefully stands to change.

As things stand, uncopyable ranges aren't really a thing, and common range
idiomns rely on ranges being copyable. We'd need some major redesigning to
make uncopyable ranges work, and personally, I don't think that it's worth
the trouble.

> > And moving doesn't fix anything, since the original variable is
> > still there
> > (just in its init state, which would still be invalid to use in
> > generic code and could outright crash in some cases if you
> > tried to use it - e.g. if it were a class reference, since it
> > would then be null).
>
> Eh? A range in 'init' state should be an empty range. If you get
> a crash from that then there's a bug in that range's
> implementation, not in user code.

The range API has no requirement that the init value of a range be empty,
and any generic code which relies on such behavior is buggy. In the case of
classes, it would outright crash, because the init value would be null. I
agree that ideally the range API would require that the init state of a
range be a valid, empty range, but that's simply not how it works right now.
In order to make it work that way, we'd have to redesign the range API in at
least some respects (e.g. getting rid of save and making it illegal for
classes to be forward ranges).

> > So, code that does a move could accidentally use the original
> > range after the move and have bugs just like code that copies
> > the range has bugs if the original is used after the copy has
> > been made. So, the rule of thumb is not that you should avoid
> > copying ranges. It's that once you've copied a range, you
> > should then use only the copy and not the original.
>
> That is not true. Copying of forward ranges is absolutely fine.
> It's what the current `save()` primitive is supposed to do. It's
> the copying of input ranges should just be rejected, statically.

As things stand, it is _not_ true that it's safe to copy forward ranges and
then use the original. Sure, it will work for some ranges, but for others it
won't. The entire reason that save exists is for ranges that are classes,
because copying them does not result in an independent range. The range API
does not require that copying a range result in an independent copy. It's
not even guaranteed that copying a range that's a struct will result in an
indpendent copy. Depending on the range type, copying it could result in an
independent copy, or it could result in a reference to to the original
range, or it could even result in part of the state being copied and part of
it being a reference (e.g. if the current front is stored directly in the
range, but the iteration state is stored by reference). If you rely on
copying a range resulting in an independent copy, you will have buggy code -
even if it's a forward range. The _only_ way that the range API specifies
that you can get an independent copy of a range is to use save, and generic
code should never rely on any other mechanism for that.

Now, ideally, we'd get rid of save and require that copying a forward range
result in an independent copy (which would then require that a class be
wrapped in a struct to be a range), but that's simply not how the current
range API works. Because the current range API does not make any guarantees
about the semantics of copying a range, generic code cannot assume that
those semantics are the same as save. As such, if you want an independent
copy of a forward range in generic code, you must use save, or the code will
be buggy. Similarly, generic code cannot use the original range after it's
been copied (unless it simply never does anything with the copy), because
mutating the copy may or may not mutate the original, and the original may
or may not even be in a valid state if the copy is mutated.

With non-generic code, you can rely on the behaviors of specific ranges and
what will happen when you copy them (e.g. not bothering to call save when
passing strings around), but with generic code, that's not true. And there's
plenty of D code out there that works correctly with a specific range type
but which would fail miserably if it were used 

Re: called copy constructor in foreach with ref on Range

2020-06-22 Thread Jonathan M Davis via Digitalmars-d-learn
On Monday, June 22, 2020 3:38:02 PM MDT Paul Backus via Digitalmars-d-learn 
wrote:
> On Monday, 22 June 2020 at 21:33:08 UTC, H. S. Teoh wrote:
> > Jonathan is coming from the POV of generic code.  The problem
> > with move leaving the original range in its .init state isn't
> > so much that it will crash or anything (even though as you said
> > that does indicate a flaw in the range's implementation), but
> > that the semantics of generic code changes in subtle ways. For
> > example:
> >
> > [...]
>
> Seems to me like adding some unit tests with non-copyable input
> ranges to Phobos could expose a number of latent bugs.

At this point, non-copyable ranges really aren't a thing. foreach does not
support them and neither does Phobos. isInputRange doesn't actually reject
them, but they don't really work in practice, and it's unlikely that you're
going to find much in Phobos that happens to work with them. isForwardRange
does outright reject them though.

- Jonathan M Davis





Re: Temporary File Creation

2020-06-22 Thread Jonathan M Davis via Digitalmars-d-learn
On Monday, June 22, 2020 3:46:57 PM MDT Per Nordlöw via Digitalmars-d-learn 
wrote:
> Has anybody written a procedure for creating a temporary file in
> a race-free manner?
>
> And why has such a procedure not already been added to std.file
> when std.file.tempDir has?
>
> See: https://dlang.org/library/std/file/temp_dir.html

I created a PR for one a while back that resulted in a surprisingly large
amount of arguing. IIRC, there were some tweaks I still needed to make to
get it merged, but I keep forgetting to get back to it.

- Jonathan M Davis






Re: called copy constructor in foreach with ref on Range

2020-06-22 Thread Jonathan M Davis via Digitalmars-d-learn
On Monday, June 22, 2020 3:10:28 PM MDT kinke via Digitalmars-d-learn wrote:
> On Monday, 22 June 2020 at 20:51:37 UTC, Jonathan M Davis wrote:
> > [...]
>
> That's why I put the struct in parantheses. Moving a class ref
> makes hardly any sense, but I've also never written a *class* to
> represent a range. Moving is the no-brainer solution for
> transferring ownership of struct ranges and invalidating the
> original instance.

Invalidating the instance doesn't actually prevent it from being misused
though. At best, the fact that you moved the instance rather than copying it
makes it more likely that accidentally using the instance will cause more
extreme bugs or crash. The core issue that the original is potentially
invalid but can still be used exists whether you copy the range or move it.

Also, since the issue here is generic code, you have to take classes into
account and cannot assume that the range is a struct. True, it's usually a
bad idea to use classes for ranges, and ideally, we'd alter the range API so
that classes weren't valid ranges, but the reality of the matter is that
they are, and generic code has to take that into account.

- Jonathan M Davis





Re: called copy constructor in foreach with ref on Range

2020-06-22 Thread Jonathan M Davis via Digitalmars-d-learn
On Monday, June 22, 2020 1:41:34 PM MDT kinke via Digitalmars-d-learn wrote:
> On Monday, 22 June 2020 at 19:03:44 UTC, Jonathan M Davis wrote:
> > in practice, that means that generic code cannot use a range
> > once it's been copied
>
> Translating to a simple rule-of-thumb: never copy a (struct)
> range, always move.

You're unlikely to find much range-based code that does that and there
really isn't much point in doing that. Again, copying isn't the problem.
It's using the original after making the copy that's the problem. And moving
doesn't fix anything, since the original variable is still there (just in
its init state, which would still be invalid to use in generic code and
could outright crash in some cases if you tried to use it - e.g. if it were
a class reference, since it would then be null). So, code that does a move
could accidentally use the original range after the move and have bugs just
like code that copies the range has bugs if the original is used after the
copy has been made. So, the rule of thumb is not that you should avoid
copying ranges. It's that once you've copied a range, you should then use
only the copy and not the original.

- Jonathan M Davis





Re: called copy constructor in foreach with ref on Range

2020-06-22 Thread Jonathan M Davis via Digitalmars-d-learn
On Monday, June 22, 2020 10:59:45 AM MDT kinke via Digitalmars-d-learn wrote:
> If copying a range is considered to be generally unsafe and a
> common pitfall (vs. the save() abomination), maybe range-foreach
> shouldn't allow any lvalue ranges in the 1st place, thus not
> making any copies and forcing the user to specify some rvalue (as
> returned by `range.save()`, or `move(range)` if destructive
> iteration is indeed intended).

Copying ranges isn't a problem. Almost all range-based code copies ranges
quite a bit (e.g. almost all range-based functions take a range by value,
and chaining range-based function calls wouldn't work if it didn't). Rather,
it's using the original after a copy has been made that's a problem (at
least in generic code), because the semantics of copying aren't part of the
range API and can vary wildly depending on the type.

Really, the issues with copying were not properly taken into account when
the range API was created, and mistakes were made. If we were to rework the
range API at this point, we would probably get rid of save and require that
copying be equivalent to save for forward ranges (which would then make it
illegal for classes to be forward ranges without wrapping them in a struct).
That would fix the problem for forward ranges, but basic input ranges can't
have those semantics, or they could be forward ranges, so exactly what the
correct solution would be is debatable, and if generic code operates on both
basic input ranges and forward ranges, the copying semantics won't always be
the same, which is the problem we have now. So, if we were to rework the
range API, it's one of the open problems that needs to be sorted out.

Regardless, as things stand, because copying a range can have reference
semantics, value semantics, or something in between, in practice, that means
that generic code cannot use a range once it's been copied, because the
semantics will vary from type to type. The copy can be used (and most
range-based code relies on that), but the original needs to be left alone.
Non-generic code has more leeway, because it can rely on the behavior of
specific range types, but with generic code, you have to be careful. And
foreach is just one of the places where the issue of not using the original
after making a copy comes up.

- Jonathan M Davis





Re: called copy constructor in foreach with ref on Range

2020-06-22 Thread Jonathan M Davis via Digitalmars-d-learn
On Sunday, June 21, 2020 2:25:37 PM MDT kinke via Digitalmars-d-learn wrote:
> A foreach over a custom range makes an initial copy, so that the
> original range is still usable after the foreach (not empty).

No, it's not so that the range will be useable afterwards. In fact, for
generic code, you must _never_ use a range again after passing it to
foreach, because the copying semantics of ranges are not specified, and you
can get radically different behavior depending on what the copying semantics
of a range are (e.g. if it's a class, then it's just copying the reference).
In general, you should never use a range after it's been copied unless you
know exactly what type of range you're dealing with and what its copying
behavior is. If you want an independent copy, you need to use save.

The reason that foreach copies the range is simply due to how the code is
lowered. e.g.

foreach(e; range)
{
}

essentially becomes

for(auto r = range; !r.empty; r.popFront())
{
auto e = r.front;
}

And the fact that a copy is made is likely simply a result of it mimicking
what happens with arrays. Either way, you should never be doing something
like

foreach(e; range)
{
}

auto v = range.front;

in generic code. It needs to be

foreach(e; range.save)
{
}

auto v = range.front;

instead.

- Jonathan M Davis





Re: final switch problem

2020-06-13 Thread Jonathan M Davis via Digitalmars-d-learn
On Saturday, June 13, 2020 10:22:39 AM MDT John Chapman via Digitalmars-d-
learn wrote:
> On Saturday, 13 June 2020 at 15:33:55 UTC, Boris Carvajal wrote:
> > On Saturday, 13 June 2020 at 09:02:21 UTC, John Chapman wrote:
> >> Is this a bug or have I made a mistake? This worked a few days
> >> ago and I haven't changed my setup since then.
> >
> > https://issues.dlang.org/show_bug.cgi?id=19548
> >
> > Your code triggers it by using "-debug" option on
> > https://run.dlang.io/ using DMD
>
> Hmm, compiling with -release makes it work. Not a huge issue,
> I'll just avoid final switches in debug mode until it's fixed.
> Thanks.

Just be aware that that removes the code that gets generated which throws a
SwitchError if the wrong value is passed to the final switch. So, instead of
getting a SwitchError thrown, you'll get who-knows-what weird behavior
happening if you have a bug with what you pass to the the switch (and of
course, you lose assertions in general). It does sound like the problem
that's resulting in the compilation error relates to the invisible default
case that gets generated without -release though.

- Jonathan M Davis





Re: Should opIndex take precedence over pointer indexing?

2020-06-12 Thread Jonathan M Davis via Digitalmars-d-learn
On Friday, June 12, 2020 5:12:35 PM MDT claptrap via Digitalmars-d-learn 
wrote:
> struct Foo
> {
>  float* what;
>  float opIndex(size_t idx) { return what[idx]; }
> }
>
> Foo* foo;
> float x = foo[idx]; // ***
>
>
> *** (68): Error: cannot implicitly convert expression
> `foo[cast(ulong)idx]` of type `Foo` to `float`
>
> IE, pointer indexing of 'foo' takes precedence over the opIndex
> of foo. Is this expected?

Yes, it's expected behavior. . is the only operator which implicitly
dereferences a pointer.

- Jonathan M Davis





Re: What is the current stage of @property ?

2020-06-10 Thread Jonathan M Davis via Digitalmars-d-learn
On Wednesday, June 10, 2020 3:41:54 PM MDT H. S. Teoh via Digitalmars-d-learn 
wrote:
> On Wed, Jun 10, 2020 at 08:24:19PM +, Vinod K Chandran via Digitalmars-
d-learn wrote:
> > Hi all,
> > I read in an old thread that authors of D wants to eliminate
> > @property. I just roughly read the big thread bu couldn't find a
> > conclusion. After all that thread is a 48 page longer jumbo thread. So
> > out of curiosity, i am asking this. What is the current state of
> > @property ? Is it deprecated ?
>
> It's stuck in limbo, like many things that people just cannot agree on.
> There are a few places where it's needed (like satisfying the range API,
> which implicitly checks for it), but for the most part, you can just
> ignore it, it doesn't really make a big difference.  Life goes on.

Yeah. There was a ton of arguing about what to do with it in the past, with
the intention at one point being that @property functions would always be
called without parens, and all function without @property would always be
called with parens, but there was never full agreement on it, and once UFCS
became widespread, the idea of requiring parens on non-@property functions
became a lot less popular, because many people don't like the idea of having
empty runtime argument parens on a function call after the template argument
parens - e.g. foo.map!(a => a * 42)() vs foo.map!(a => a * 42). Ultimately,
@property was never completed, and it's sort of just lingered on.

As I understand it, @property currently does exactly two things:

1. Become an attribute on the function which affects the mangling and can be
queried my metaprogramming. IIRC, one result of this is that you can't
overload a function that has @property with one that doesn't. Most traits in
Phobos do not check for @property, but it's possible that some do. At one
point, isForwardRange checked for it for save (which never made sense, since
save doesn't act like a property), but that was removed.

2. Screw with the type of the function so that traits will mostly claim that
its type is the return type of the function rather than a function that
returns that type. This can cause some subtle and annoying problems with
metaprogramming.

Now, there is one ambiguity that @property was supposed to solve that was
never solved, and that's a property function that returns a callable. Right
now, you're forced to use double parens (one set being the optional parens,
and the other being the set to call the return value), whereas if it were
truly treated as a property function, then the first set of parens would
call the return value. As such, you can't really have a property function
that returns a callable (whether @property is used or not). So, even if we
were to actually get rid of @property, we might want to keep it for that
case, but since it's never actually be made to fix that case, such a change
would sadly be a breaking change.

As things stand, @property has no real practical purpose but frequently gets
used to indicate that it's the intention of a function's author for it to be
used as if it were a variable. I suspect that it's also frequently
misunderstand that it's required if you want to call a function without
parens. So, you're likely to see @property in quite a lot of D code, but
ultimately, all it's really doing is serving as documentation of the
author's intent and screwing up metaprogramming.

- Jonathan M Davis





Re: Distinguish between a null array and an empty array

2020-05-24 Thread Jonathan M Davis via Digitalmars-d-learn
On Sunday, May 24, 2020 6:12:31 AM MDT bauss via Digitalmars-d-learn wrote:
> Is there a way to do that?
>
> Since the following are both true:
>
> int[] a = null;
> int[] b = [];
>
> assert(a is null);
> assert(!a.length);
>
> assert(b is null);
> assert(!b.length);
>
> What I would like is to tell that b is an empty array and a is a
> null array.

There really is no difference between null and []. The compiler will use the
exact same value for both, since it makes no sense to allocate anything for
an empty array. If you want an array that's empty but non-null, then you'll
have to create a non-empty array and then slice it or set its length to 0 so
that it actually has a value for its ptr property other than null.

Regardless, == will never differentiate between a null and empty array,
because it's going to compare the lengths of the arrays and then compare the
individual elements if the lengths are the same. And with a length of 0,
there's no reason to ever check the elements, so the ptr value is
irrelevant. If you want to check whether an array is actually null, then you
need to use the is operator or check the ptr property explicitly.

And of course, the normal warnings apply that trying to differentiate
between null and empty arrays in D is almost certainly something that you
shouldn't be doing. You can do it if you're _very_ careful, but it's almost
always a terrible idea. As dumb as it arguably is, wrapping the array in a
Nullable would be _far_ less error-prone if you need to differentiate
between an array that's empty and not having a value.

- Jonathan M Davis





Re: opEquals @safe is ignored

2020-05-24 Thread Jonathan M Davis via Digitalmars-d-learn
On Sunday, May 24, 2020 2:57:28 AM MDT Luis via Digitalmars-d-learn wrote:
> Lets take this example code (https://run.dlang.io/is/Vkpx9j) :
>
> ´´´D
> import std;
>
> void main()
> {
> }
>
> class ExampleC
> {
>int x;
>this (int x) @safe
>{
>  this.x = x;
>}
>
>override bool opEquals(Object o) const @trusted
>{
>  if (ExampleC rhs = cast(ExampleC)o) {
>return this.x == rhs.x;
>  }
>  return false;
>}
> }
>
> @safe unittest
> {
>  auto c = new ExampleC(1);
>  assert(c != new ExampleC(23));
> }
> ´´´
>
> dmd ignores @trusted or @safe on opEquals, throwing this error :
>
> onlineapp.d(27): Error: @safe function
> onlineapp.__unittest_L24_C7 cannot call @system function
> object.opEquals
>
> An override @system or @trusted function can't be @safe, or I it
> a bug ?
>
> Also, how will this be affected by DIP1028 ?

The core problem is that Object does not have attributes on any of its
functions - including opEquals. And in the case of opEquals, the problem is
one layer worse, because == gets lowered to the free function opEquals which
in turn calls opEquals on the class reference itself (after doing additional
checks like that the reference isn't null and that you're not comparing a
reference to itself), and that free function has no attributes. In fact, the
only reason that using == on const objects works is because of a hack in
druntime that casts away const (which means that it's technically possible
and even downright trivial to break the type system by mutating a const
class object within opEquals).

And really, the only fix for this is to remove opEquals, opCmp, toHash, and
toString from Object, since no matter which set of attributes you pick,
there will be problems. With the addition of templates, it's no longer
necessary for any of those functions to even be on Object, and it really
doesn't make sense for them to be there, but they've been there since long
before templates or attributes were added to the language.

It was decided years ago that those functions should be removed from Object,
but _how_ to do it with minimal code breakage is a thorny problem, and it
hasn't really been a priority. The currently proposed solution (which has a
work-in-progress DIP IIRC) is to introduce a new root class to the language
below Object which has _nothing_ on it (the current name for that root class
being ProtoObject). Object would then continue to be the default base class
of any class, but it would then be derived from ProtoObject, and best
practice at that point would really be to explicitly derive your class from
ProtoObject (with Object being left in place pretty much just to avoid
breaking existing code). However, the DIP for ProtoObject has yet to be
fully sorted out, and it definitely hasn't been accepted yet, so it can't
yet fix the problem.

So, for now, you're basically forced to use @trusted when comparing class
references in @safe code. It's annoying, but because Object's opEquals is
@system, we're kind of stuck at the moment.

- Jonathan M Davis






Re: Asserting that a base constructor is always called

2020-05-24 Thread Jonathan M Davis via Digitalmars-d-learn
On Sunday, May 24, 2020 12:38:46 AM MDT Tim via Digitalmars-d-learn wrote:
> Oh right. I mean it makes sense but I got confused when super()
> is valid syntax. Why would you need to call the super constructor
> when it's called automatically?

1. If you wanted to run any code before calling the base class constructor.
e.g. depending on the arguments to the derived class constructor, you could
call one base constructor, or you could call a different base class
constructor. You could even have the base class constructor call a virtual
function, and what that virtual function did depended on something you had
set in the derived class prior to calling the base class constructor (since
unlike C++, it's safe to call virtual functions from within constructors).

2. If the base class constructor is not a default constructor, then you have
to explicitly call it, because it has to be passed arguments.

- Jonathan M Davis





Re: Asserting that a base constructor is always called

2020-05-23 Thread Jonathan M Davis via Digitalmars-d-learn
On Saturday, May 23, 2020 4:43:04 PM MDT Tim via Digitalmars-d-learn wrote:
> It is but I want to make sure for other cases in the future where
> I create a new class that inherits from GameObject. This was I
> can avoid future bugs by ensure that all classes in the future
> that inherit from GameObject, call it's constructor

The base class constructor will _always_ be called. There's no need to worry
about it. Even if you hadn't declared one, the compiler will provide a
default constructor, because classes have to have a constructor. And if you
had declared a constructor in your base class but no default constructor,
then you'd get a compile-time error if your derived class' constructor
didn't explicitly call it.

- Jonathan M Davis





Re: any chance to get it working on windows xp?

2020-05-18 Thread Jonathan M Davis via Digitalmars-d-learn
On Sunday, May 17, 2020 11:36:01 PM MDT Mike Parker via Digitalmars-d-learn 
wrote:
> Unfortunately, the minimum Windows version "officially" supported
> is Windows 7:
>
> https://forum.dlang.org/post/ktfgps$2ghh$1...@digitalmars.com
>
> With no testing on XP, you are bound to run into difficulties
> trying to use the tools there. So yeah, your best bet is using a
> compiler version that works and see if building dub from source
> makes a difference. If you can't get dub to work, then you'll
> want to look into using rdmd, which has shipped with dmd for
> years now, or perhaps makefiles.

I'm pretty sure that Phobos (and possibly druntime) use Windows API calls
that do not exist on XP and have done so for years now. You might get lucky
and get some stuff to work on XP, but not everything will work. The only way
to guarantee that it will work is to use a compiler version old enough that
XP was still supported (which probably means grabbing one from the 2013
timeframe given the date of Walter's post, though it may be possible to dig
through the changelog and find the exact version that Windows XP support was
officially dropped). However, you're not going to be able to pull in much of
anything with dub that way, because the older compiler likely won't be able
to compile the newer code, and I don't know if a version of dub that old
will ever work with the current dub repository.

- Jonathan M Davis





Re: Retrieve the return type of the current function

2020-05-05 Thread Jonathan M Davis via Digitalmars-d-learn
On Tuesday, May 5, 2020 11:11:53 AM MDT learner via Digitalmars-d-learn 
wrote:
> On Tuesday, 5 May 2020 at 16:41:06 UTC, Adam D. Ruppe wrote:
> > typeof(return)
>
> Thank you, that was indeed easy!
>
> Is it possible to retrieve also the caller return type? Something
> like:
>
> ```
> int foo() {
>  return magic();
> }
>
> auto magic(maybesomedefaulttemplateargs = ??)() {
>  alias R = __traits(???); // --> int!
> }
> ```
>
> Mixin templates maybe?

A function is compiled completely independently of where it's used, and it's
the same regardless of where it's used. So, it won't ever have access to any
information about where it's called unless it's explicitly given that
information.

A function template will be compiled differently depending on its template
arguments, but that still doesn't depend on the caller at all beyond what it
explicitly passes to the function, and if the same instantiation is used in
multiple places, then that caller will use exactly the same function
regardless of whether the callers are doing anything even vaguely similar
with it.

So, if you want a function to have any kind of information about its caller,
then you're going to have to either explicitly give it that information via
a template argument or outright generate a different function with a string
mixin every time you use it. So, you could do something like

auto foo(T)(int i)
{
...
}

string bar(string s, int i)
{
return!string(i);
}

or

string bar(string s, int i)
{
return!(typeof(return))(i);
}

but you're not going to be have foo figure out anything about its caller on
its own.

- Jonathan M Davis





Re: How can I check if an element is iterable?

2020-05-03 Thread Jonathan M Davis via Digitalmars-d-learn
On Sunday, May 3, 2020 2:54:17 PM MDT Marcone via Digitalmars-d-learn wrote:
> On Sunday, 3 May 2020 at 20:46:30 UTC, Adam D. Ruppe wrote:
> > On Sunday, 3 May 2020 at 20:21:24 UTC, Marcone wrote:
> >> How can I check if a variable is iterable?
> >
> > Every variable has a type. You can get it with typeof(varaiable)
>
> I need in runtime.

Then store that information in an enum or variable. I don't know why you'd
care about it at runtime though. D is statically typed, so the code that
gets compiled would be completely different depending on the type, and you'd
have to determine at compile time whether you were going to try to iterate
over it or not. At most, you'd have something like a variant where you had
to cast it to a particular type, and which branch you took would be decided
at runtime. But even then, all of the code related to iteration would have
to be dealt with at compile time, so all of the checks for whether a type
was iteraterable or not would be done at compile time.

- Jonathan M Davis





Re: Help, what is the code mean?

2020-04-28 Thread Jonathan M Davis via Digitalmars-d-learn
On Monday, April 27, 2020 9:52:32 AM MDT drug via Digitalmars-d-learn wrote:
> 27.04.2020 18:28, data pulverizer пишет:
> > I'm probably not the first person to say this but. Isn't @trusted an odd
> > label to give unsafe functions and open to abuse by unscrupulous
> > programmers? It almost says "nothing to see, this here piece of code is
> > a-ok". Shouldn't it be explicitly labelled as @unsafe?
>
> It says "this piece of code is verified by its author manually so you
> (the compiler) can trust it is @safe"

Exactly. @trusted isn't about marking something as not being memory safe.
The compiler already treats anything as not being memory safe if it can't
verify that it's memory safe. It's about the programmer telling the compiler
that they've verified that it's memory safe even though the compiler
couldn't. The code that neither the programmer nor the compiler has verified
to be memory safe is @system. So, if we had the attribute @unsafe, it would
have been instead of @system, not @trusted.

And ultimately, @trusted is not about telling anyone that there's "nothing
to see." If anything, it's the opposite. @trusted code is the primary place
that has to be examined when you have a memory bug in your code (or think
that you have one). Barring bugs in the compiler, it should not be possible
for @safe code to do anything that's memory unsafe, so when looking for
memory safety bugs, it's the @trusted code that has to be examined to make
sure that it actually is memory safe and that the programmer didn't use
@trusted correctly.

- Jonathan M Davis






Re: make dub use my version of dmd

2020-04-16 Thread Jonathan M Davis via Digitalmars-d-learn
On Thursday, April 16, 2020 12:41:14 PM MDT Steven Schveighoffer via 
Digitalmars-d-learn wrote:
> On 4/16/20 2:28 PM, Steven Schveighoffer wrote:
> > OK, I thought, just put it in ~/bin, and run it from there. Doesn't
> > work, now it looks in ~/bin (where there is no compiler), and fails.
>
> I wish I could delete this idiotic post.
>
> I had a broken link to a dmd compiler in ~/bin. Removing that now it
> works.
>
> Carry on everyone...

LOL. Well, things like that happen to us all from time to time. I don't even
want to think about how much of my life I've wasted because of my own
stupidity with stuff like this...

I _was_ going to suggest just building dub yourself, since that's what I
have on my system (with it in ~/bin), and I've never had this problem, but
clearly, you found the issue. :)

- Jonathan M Davis





Re: No implicit opOpAssign for structs with basic types?

2020-04-04 Thread Jonathan M Davis via Digitalmars-d-learn
On Saturday, April 4, 2020 4:22:29 AM MDT Robert M. Münch via Digitalmars-d-
learn wrote:
> D doesn't have implicit operators for structs?
>
> struct S {float a, float b};
> S a = {1, 5};
> S b = {2, 5);
>
> a += b;
> Error: a is not a scalar, it is a S
>
> So I really have to write an overloaded operator for such cases?

You could just use the constructor syntax. e.g.

S a = S(1, 5);
S b = S(2, 5);

- Jonathan M Davis






Re: OR in version conditional compilation

2020-03-20 Thread Jonathan M Davis via Digitalmars-d-learn
On Friday, March 20, 2020 4:33:58 PM MDT jxel via Digitalmars-d-learn wrote:
> On Friday, 20 March 2020 at 21:03:55 UTC, Jonathan M Davis wrote:
> > On Wednesday, March 18, 2020 10:23:26 AM MDT IGotD- via
> >
> > Digitalmars-d-learn wrote:
> >> I have not seen any example where version has several OR
> >> matches.
> >>
> >> Example idiom:
> >>
> >> version(X86_64 || X86)
> >> {
> >>
> >> }
> >> else version(ARM || Thumb)
> >> {
> >>
> >> }...
> >>
> >> you get the idea. So is this possible at all or do you have to
> >> duplicate the code for each version identifier despite they
> >> are equal for many version identifiers?
> >
> > To add to what the others have said, the reasons that Walter is
> > against having boolean conditions in version statements have to
> > do with how using boolean conditions in #ifdefs in C/C++ has
> > historically been a big source of bugs. So, disallowing it
> > helps prevent certain classes of bugs. It's why druntime
> > duplicates most C declarations across platforms.
>
> What kind of bugs are those?

There are a variety of issues that come from it, and it's been discussed at
length in past threads (and much better than I'll explain here, I'm sure),
but the main issues stem from the fact that when you're writing code that is
platform-dependent but using it on multiple platforms, it can become really
easy to break stuff as the code is altered over time. e.g. you can make a
change that works perfectly fine on the platform that you're developing on
without realizing that it won't work on the other platforms sharing the same
#ifdef block, and while you should be testing such code on all supported
platforms, it often doesn't happen, and even when it does happen, it's
usually much further down the line after many more changes have been made.
It would be highly abnormal for someone to test what they're working on on
all of the relevant platforms as they're working on it.

It also can get really tricky to avoid subtle bugs once you start having
more complex boolean expressions and/or have nested #ifdefs. That's where
things tend to get _really_ bad. Another big problem comes in when you just
assume that the code from one platform will work with another and alter
existing #ifdefs to be used on the new platform. Suddenly code that was only
supposed to be used on one platform is being used on multiple, and it can
cause big problems depending on what that code actualy does and what
assumptions were made when writing it. By simply following the basic idiom
of

version(A)
{
}
else version(B)
{
}
else
static assert(false, "Platform not supported");

you avoid problems that stem from code being used an a platform that it's
not intended for, and you get compilation errors if you try to use code on a
platform that it wasn't made to work on yet. It can result in more code
duplication than many people like, but it's ultimately less error-prone.
Either way, in general, it's far better to design your code such that as
little of it as possible is platform-dependent and that those parts that are
are well-encapsulated.

- Jonathan M Davis





Re: OR in version conditional compilation

2020-03-20 Thread Jonathan M Davis via Digitalmars-d-learn
On Wednesday, March 18, 2020 10:23:26 AM MDT IGotD- via Digitalmars-d-learn 
wrote:
> I have not seen any example where version has several OR matches.
>
> Example idiom:
>
> version(X86_64 || X86)
> {
>
> }
> else version(ARM || Thumb)
> {
>
> }...
>
> you get the idea. So is this possible at all or do you have to
> duplicate the code for each version identifier despite they are
> equal for many version identifiers?

To add to what the others have said, the reasons that Walter is against
having boolean conditions in version statements have to do with how using
boolean conditions in #ifdefs in C/C++ has historically been a big source of
bugs. So, disallowing it helps prevent certain classes of bugs. It's why
druntime duplicates most C declarations across platforms.

In general, I think that Walter's approach here is probably the right one,
but it can certainly be annoying in cases where you have to duplicate code
that actually is the same rather than being subtly different, and a lot of
people dislike it enough that they use workarounds like those discussed in
this thread.

And of course, some of the same bugs that come up with #ifdefs come up with
static ifs in generic code (particularly with range-based code that changes
what it's doing based on arange's type), but at least those can be found
with thorough tests on a single platform, whereas version-related bugs tend
to require that you run thorough tests on each platform.

- Jonathan M Davis





Re: std.datetime & timzone specifier: 2018-11-06T16:52:03+01:00

2020-03-09 Thread Jonathan M Davis via Digitalmars-d-learn
On Sunday, March 8, 2020 11:19:33 PM MDT tchaloupka via Digitalmars-d-learn 
wrote:
> On Sunday, 8 March 2020 at 17:28:33 UTC, Robert M. Münch wrote:
> > On 2020-03-07 12:10:27 +, Jonathan M Davis said:
> >
> > DateTime dt =
> > DateTime.fromISOExtString(split("2018-11-06T16:52:03+01:00",
> > regex("\\+"))[0]);
> >
> > IMO such a string should be feedable directly to the function.
>
> You just need to use SysTime.fromISO*String functions for that,
> as DateTime does't work with timezones, SysTime do.

Exactly. DateTime does not support timezones or fractional seconds. If you
want that, then use SysTime. And if you want to get a DateTime from a string
such as "2018-11-06T16:52:03+01:00", then just use SysTime's
fromISOExtString and convert the result to DateTime. e.g.

auto dt =
cast(DateTime)SysTime.fromISOExtString("2018-11-06T16:52:03+01:00"");

The documentation for the from*String functions say what they support.

- Jonathan M Davis






Re: std.datetime & timzone specifier: 2018-11-06T16:52:03+01:00

2020-03-07 Thread Jonathan M Davis via Digitalmars-d-learn
On Saturday, March 7, 2020 2:43:47 AM MST Robert M. Münch via Digitalmars-d-
learn wrote:
> It looks like std.datetime is not anticipating the +1:00 part of a date
> like: "2018-11-06T16:52:03+01:00"
>
> Those dates are used all over on the internet and I'mm wondering why
> it's not supported. Any reason? Is this +01:00 not ISO conforming?

I take it that you're asking why you don't get the time zone as part of the
string when you call one of the to*String functions? If so, then you're
using LocalTime for your time zone. LocalTime does not print out the UTC
offset as part of the ISO string, whereas other time zones types do (or z in
the case of UTC). This is because the ISO spec says that when the time zone
is not provided, it's assumed to be local time. When I wrote it, I therefore
made it so that the time zone was not put at the end of the string when it
was local time. I don't remember now if I thought that that was required, or
if I just did it, because it corresponded to what happens when you read an
ISO or ISO extended string without a time zone, but at this point, I think
that it was a mistake. Yes, the spec requires that the time zone be treated
as local time when it's not present, but that doesn't mean that the time
zone can't be printed when it's local time. And in general, generating an
ISO string without a time zone is asking for bugs, since the time then won't
match if it's read in on a computer with a different time zone (though it
isn't likely to matter much if it's just printed out).

Unfortunately, I'm not sure that it's reasonable to fix it at this point,
since there's bound to be software that relies on the current behavior.
Given the current behavior, I think that it's usually best to call toUTC
before calling any of the to*String functions unless you're just printing to
a log or something, and there's no real risk of the string being intepreted
as a time value by a program in the future.

- Jonathan M Davis






Re: Difference between range `save` and copy constructor

2020-02-16 Thread Jonathan M Davis via Digitalmars-d-learn
On Sunday, February 16, 2020 12:22:01 PM MST Paul Backus via Digitalmars-d-
learn wrote:
> On Sunday, 16 February 2020 at 18:11:11 UTC, Jonathan M Davis
>
> wrote:
> > Either way, generic code should never be using a range after
> > it's been copied, and copying is a key part of how idiomatic,
> > range-based code works in D.
>
> "Copy and then never use the original again" is conceptually the
> same thing as "move", right? In which case, generic code can
> accommodate non-copyable ranges *and* more clearly communicate
> its intent by using `move` instead of a naked copy.

We already have enough of a mess with save without making things even worse
by trying to add moves into the mix. Also, non-copyable ranges have never
really been a thing, and I really don't want to see things complicated even
further trying to support such an uncommon use case. There are too many
weird corner cases with ranges as it is.

- Jonathan M Davis





Re: Difference between range `save` and copy constructor

2020-02-16 Thread Jonathan M Davis via Digitalmars-d-learn
On Sunday, February 16, 2020 10:53:36 AM MST Paul Backus via Digitalmars-d-
learn wrote:
> On Sunday, 16 February 2020 at 17:10:24 UTC, Jonathan M Davis
>
> wrote:
> > On Sunday, February 16, 2020 7:29:11 AM MST uranuz via
> >
> >> This is working fine with disabled postblit...
> >> import std;
> >>
> >> struct SS
> >> {
> >>
> >>  @disable this(this); // Disabled copy
> >>
> >>  bool _empty = false;
> >>
> >>  bool empty() @property {
> >>
> >>  return _empty;
> >>
> >>  }
> >>
> >>  void popFront() {
> >>
> >> _empty = true;
> >>
> >>  }
> >>
> >>  int front() @property { return 10; }
> >>
> >> }
> >>
> >>
> >> void main()
> >> {
> >>
> >>  foreach( it; SS() ) { writeln(it); }
> >>
> >> }
> >>
> >> Am I missing something?
> >
> > That code compiles, because you're passing a temporary to
> > foreach. So, the compiler does a move instead of a copy. It's
> > the difference between
> >
> > auto ss = SS();
> >
> > and
> >
> > SS ss;
> > auto ss2 = ss;
> >
> > If your main were
> >
> > void main()
> > {
> >
> > SS ss;
> > foreach( it; ss ) { writeln(it); }
> >
> > }
> >
> > then it would not compile.
>
> On the other hand, this does work:
>
>  void main()
>  {
>  SS ss;
>  foreach( it; move(ss) ) { writeln(it); }
>  }
>
> So perhaps the correct approach is to use `move` when copying
> input ranges.

Given that the way that almost all range-based functions work is to copy the
range that they're given (often then wrapping it in another range that's
returned), I don't see how it would make sense to use move outside of very
specific circumstances. If you pass a range to a function, and it's a basic
input range, then you just use the range via the return value (be it the
same range returned directly or returned within a wraper range), and if it's
a forward range, you call save before passing it to the function if you want
to be able to use the range directly again. Either way, generic code should
never be using a range after it's been copied, and copying is a key part of
how idiomatic, range-based code works in D.

And really, using move just to be able to use an uncopyable range with
foreach doesn't make a lot of sense, since if that's what you want to do,
you can always just use a normal for loop. Regardless, there isn't much
point in declaring a range type that can't be copied, since it's pretty much
only going to work with code that you write.

- Jonathan M Davis





Re: Difference between range `save` and copy constructor

2020-02-16 Thread Jonathan M Davis via Digitalmars-d-learn
On Sunday, February 16, 2020 6:52:17 AM MST uranuz via Digitalmars-d-learn 
wrote:
> It's very bad. Because there seem that when I use range based
> algorithm I need to take two things into account. The first is
> how algrorithm is implemented. If it creates copies of range
> inside or pass it by reference. And the second is how the range
> is implemented if it has value or reference semantics. So every
> time I need to look into implementation and I can't rely on API
> description in most of the cases. In a lot of cases Phobos uses
> value semantics. But there are cases where I want the range
> actually be consumed, but it's not. And the other problemme is
> when algorithm expects range to have value semantics, but it's
> not. So it's a buggy mess that it's hard to think about. In
> trivial cases this is working although. But in more complex cases
> it's simplier to implement some algorithms by own hands so that
> it would work as I expect it myself rather that thinking about
> all these value-ref-range mess. But still can't say that I
> implement it correctly, because range specification actually
> sucks as yo say.
> It's just horrible

The current situation is definitely not ideal, but it can be used quite
consistently. Just follow the rule that once you copy a range, you do not
use that range ever again unless you assign it a new value. So, if you do
something like

auto result = r.find(e);

then you should not be using r again unless you assign it a new value.
e.g.

r = r.find(e);
r.popFront();

auto result = r.find(e);
r = otherRange;
r.popFront();

would be fine, because r was assigned a new value, whereas

auto result = r.find(e);
r.popFront();

should never happen in your code unless you know that typeof(r) is a type
where copying it is equivalent to calling save on it. In generic code, that
means that you should never use a range again after passing it to a function
unless you assign it a new value, or that function accepted the argument by
ref (which almost no range-based functions do).

If you want to be able to use the range again after passing it to a function
without assigning it a new value, then you need to call save so that you
pass an independent copy. e.g.

auto result = r.find.save(e);
r.popFront();

The only copying going on here is with save, so there's no problem, whereas
if r were passed to find directly, the behavior is implementation-dependent
- hence why you should not be using a range after it's been copied (which
includes passing it to a function).

The only time that generic code can reuse a range after passing it to a
function is if that function accepts its argument by ref, which almost no
range-based code does. Far more frequently, it returns the result as a
wrapper range, in which case, you can't use the original range any longer
unless you used save when calling the function or if the code is not generic
and you're coding based on the specific behavior of that particular range
type - which usually isn't something that code should be doing.

By no means do I claim that the status quo here is desirable, but if you
just follow the simple rule that you don't ever use a range once it's been
copied (unless that copy came from save), then you shouldn't run into
problems related to the fact that different ranges have different copying
semantics unless the function that you're calling is buggy.

If you're going to run into a bug along those lines though, it's likely
going to be because a function didn't call save when it was supposed to, and
it was only tested with range types where copying them is equivalent to
save. That's why it's important to test range-based code with both range
types where copying them is equivalent to save and range types which are
full-on reference types (and thus copying just results in another
reference). In general though, any range that is a forward range should have
copying it be equivalent to save, and using reference types for forward
ranges tends to be inefficient and error-prone even if range-based functions
(especially those in Phobos) should be able to handle them correctly.

- Jonathan M Davis





Re: Difference between range `save` and copy constructor

2020-02-16 Thread Jonathan M Davis via Digitalmars-d-learn
On Sunday, February 16, 2020 7:29:11 AM MST uranuz via Digitalmars-d-learn 
wrote:
> On Sunday, 16 February 2020 at 12:38:51 UTC, Jonathan M Davis
>
> wrote:
> > On Sunday, February 16, 2020 3:41:31 AM MST uranuz via
> >
> > Digitalmars-d-learn wrote:
> >> I have reread presentation:
> >> http://dconf.org/2015/talks/davis.pdf
> >> We declare that `pure` input range cannot be `unpoped` and we
> >> can't return to the previous position of it later at the time.
> >> So
> >> logically there is no sence of copying input range at all. So
> >> every Phobos algorithm that declares that it's is working with
> >> InputRange must be tested for working with range with disabled
> >
> > A range that can't be copied is basically useless. Not only do
> > almost all range-based algorithms take their argumenst by value
> > (and thus copy them), but foreach copies any range that it's
> > given, meaning that if a range isn't copyable, you can't even
> > use it with foreach. And since many range-based algorithms
> > function by wrapping one range with another, the ability to
> > copy ranges is fundamental to most range-based code.
>
> This is working fine with disabled postblit...
> import std;
>
> struct SS
> {
>  @disable this(this); // Disabled copy
>
>  bool _empty = false;
>
>  bool empty() @property {
>  return _empty;
>  }
>
>  void popFront() {
> _empty = true;
>  }
>
>  int front() @property { return 10; }
> }
>
>
> void main()
> {
>  foreach( it; SS() ) { writeln(it); }
> }
>
> Am I missing something?

That code compiles, because you're passing a temporary to foreach. So, the
compiler does a move instead of a copy. It's the difference between

auto ss = SS();

and

SS ss;
auto ss2 = ss;

If your main were

void main()
{
SS ss;
foreach( it; ss ) { writeln(it); }
}

then it would not compile.

foreach(e; range) {...}

basically gets lowered to

for(auto __r = range; !__r.empty; __r.popFront())
{
auto e = __r.front;
...
}

So,

foreach( it; SS() ) { writeln(it); }

would become

for(auto __r = SS(); !__r.empty; __r.popFront())
{
auto it = __r.front;
writeln(it);
}

whereas

SS ss;
foreach( it; ss ) { writeln(it); }

would become

SS ss;
for(auto __r = ss; !__r.empty; __r.popFront())
{
auto it = __r.front;
writeln(it);
}

- Jonathan M Davis





Re: Difference between range `save` and copy constructor

2020-02-16 Thread Jonathan M Davis via Digitalmars-d-learn
On Sunday, February 16, 2020 3:41:31 AM MST uranuz via Digitalmars-d-learn 
wrote:
> I have reread presentation:
> http://dconf.org/2015/talks/davis.pdf
> We declare that `pure` input range cannot be `unpoped` and we
> can't return to the previous position of it later at the time. So
> logically there is no sence of copying input range at all. So
> every Phobos algorithm that declares that it's is working with
> InputRange must be tested for working with range with disabled
> copy constructor and postblit. And if it is not it means that
> this algroithm actually requires a forward range and there we
> missing `save` calls?
> Because as it was written in this presentation a range copy is
> undefined (without call to save). So it's illegal to create copy
> of range in Phobos algorithms without `save`?
> So we need a test for every algorithm that it is working with
> range with disabled copy constructor and postblit if we declare
> that we only use `save` for range copy?

A range that can't be copied is basically useless. Not only do almost all
range-based algorithms take their argumenst by value (and thus copy them),
but foreach copies any range that it's given, meaning that if a range isn't
copyable, you can't even use it with foreach. And since many range-based
algorithms function by wrapping one range with another, the ability to copy
ranges is fundamental to most range-based code.

That being said, the semantics of copying a range are not actually defined
by the range API. Whether iterating over a copy affects the original depends
on how a range was implemented. e.g. In code such as

void foo(R)(R r)
if(isInputRange!R)
{
r.popFront();
}

foo(range);

whether the range in the original range in the calling code is affected by
the element being popped from the copy inside of foo is implementation
dependent. If it's a class or a struct that's a full-on reference type, then
mutating the copy does affect the original, whereas if copying a range is
equivalent to save, then mutating the copy has no effect on the original.
And with pseudo-reference types, it's even worse, because you could end up
_partially_ mutating the original by mutating the copy, meaning that you can
get some pretty serious bugs if you attempt to use a range after it's been
copied.

This means that in practice, in generic code, you can never use a range once
it's been copied unless you overwrite it with a new value. Passing a range
to a function or using it with foreach basically means that you should not
continue to use that range, and if you want to be able to continue to use
it, you need to call save and pass that copy to the function or foreach
instead of passing the range directly to a function or foreach.

In order to fix it so that you can rely on the semantics of using a range
after it's been copied, we'd have to rework the range API and make it so
that the semantics of copying a range were well-defined, and that gets into
a huge discussion on its own.

As things stand, if you want to test range-based code to ensure that it
works correctly (including calling save correctly), you have to test it with
a variety of different range types, including both ranges where copying is
equivalent to calling save and ranges which are reference types so that
copying them simply results in another reference to the same data such that
iterating one copy iterates all copies.

- Jonathan M Davis





Re: Difference between range `save` and copy constructor

2020-02-15 Thread Jonathan M Davis via Digitalmars-d-learn
On Saturday, February 15, 2020 7:34:42 AM MST Steven Schveighoffer via 
Digitalmars-d-learn wrote:
> On 2/15/20 5:53 AM, uranuz wrote:
> > I am interested in current circumstances when we have new copy
> > constructor feature what is the purpose of having range `save`
> > primitive? For me they look like doing basicaly the same thing. And when
> > looking in some source code of `range` module the most common thing that
> > `save` does is that it use constructor typeof(this) to create a new
> > instance and use `save` on the source range:
> > https://github.com/dlang/phobos/blob/v2.090.1/std/range/package.d
> >
> > So what is conceptual difference between `save` and copy contructor of
> > range?
>
> Nothing. IMO, any time you are doing anything in save other than `return
> this;`, you shouldn't have implemented it.
>
> The original impetus for the save requirement was so that forward ranges
> could have a tangible checkable thing that allows introspection (does
> the range have a save method?).
>
> I'm not entirely sure if disabled postblit was even available at the time.
>
> The correct way to do it would be to treat ranges that can be copied
> (regardless of whether they have a copy constructor) as forward ranges,
> and treat ones that cannot be copied as input ranges.
>
> But it's hard to redo ranges like this with all existing code out there.

Actually, as I understand it, the main reason that save was introduced was
so that classes could be forward ranges. While it would be possible to use
the postblit constructor or copy constructor with structs, that obviously
won't work for classes - hence when save is required.

Personally, I think that we'd be better of simply requiring that forward
rangse be copyable and force classes that want to be forward ranges to be
wrapped by structs, but that would require reworking the range API, and it's
far from a trivial change.

In practice though, classes should almost never be used as forward ranges,
because calling save on them would requires allocating a now object, and
that gets expensive fast. As part of testing dxml, I tested it with forward
ranges that were classes in order to make sure that they were handled
correctly, and their performance was absolutely terrible in comparison to
ranges that were structs or strings.

- Jonathan M Davis





Re: Assoc array init

2020-02-04 Thread Jonathan M Davis via Digitalmars-d-learn
On Tuesday, February 4, 2020 12:52:05 AM MST JN via Digitalmars-d-learn 
wrote:
> int[int] a = [5: 7];
>
> void main()
> {
> }
>
>
> This fails because apparently [5: 7] is a "non-const expression".
> How? Why?
>
> Yes, I know I can just init in a static this() section, but that
> feels like a bad workaround.

It's a limitation of CTFE. A variable at module scope which is directly
initialized must have its value known at compile-time, and while AAs can be
_used_ at compile-time, the compiler cannot currently transfer those AAs to
runtime. That may or may not be fixed in the future (e.g. originally, it
wasn't possible to have class objects transfer from compile-time to runtime,
but at some point, that was fixed). Either way, for now, it means that if
you want to initialize an AA like the one here, you will need to use a
static constructor.

- Jonathan M Davis





Re: Empty string vs null

2020-02-04 Thread Jonathan M Davis via Digitalmars-d-learn
On Tuesday, February 4, 2020 12:33:42 AM MST mark via Digitalmars-d-learn 
wrote:
> I have just discovered that D seems to treat empty and null
> strings as the same thing:
>
> // test.d
> import std.stdio;
> import std.string;
> void main()
> {
>  string x = null;
>  writeln("x = \"", x, "\"");
>  writeln("null  = ", x == null);
>  writeln("\"\"= ", x == "");
>  writeln("empty = ", x.empty);
>  x = "";
>  writeln("\nx = \"", x, "\"");
>  writeln("null  = ", x == null);
>  writeln("\"\"= ", x == "");
>  writeln("empty = ", x.empty);
>  x = "x";
>  writeln("\nx = \"", x, "\"");
>  writeln("null  = ", x == null);
>  writeln("\"\"= ", x == "");
>  writeln("empty = ", x.empty);
> }
>
> Output:
>
> x = ""
> null  = true
> ""= true
> empty = true
>
> x = ""
> null  = true
> ""= true
> empty = true
>
> x = "x"
> null  = false
> ""= false
> empty = false
>
> 1. Why is this?

It's a side effect of how dynamic arrays in D are structured. They're
basically

struct DynamicArray(T)
{
size_t length;
T* ptr;
}

A null array has a length of 0 and ptr which is null. So, if you check
length, you get 0. empty checks whether length is 0. So, if you check
whether an array is empty, and it happens to be null, then the result is
true.

Similarly, the code which checks for equality is going to check for length
first. After all, if the lengths don't match, there's no point in comparing
the elements in the array. And if the length is 0, then even if the lengths
match, there's no point in checking the value of ptr, because the array has
no elements. So, whether the array is empty because it's null or whether
it's because its length got reduced to 0 is irrelevant.

The natural result of all of this is that D treats null arrays and empty
arrays as almost the same thing. They're treating differently if you use the
is operator, because that checks that the two values are the same bitwise.
For instance, in the case of pointers or classe references, it checks their
point values, not what they point to. And in the case of dynamic arrays,
it's comparing both the length and ptr values. So, if you want to check
whether a dynamic array is really null, then you need to use the is operator
instead of ==. e.g.

writeln(arr is null);

instead of

writeln(arr == null);

As a side note, when using an array directly in the condition of an if
statement or assertion, it's equivalent to checking whether it's _not_ null.
So,

if(arr) {...}

is equivalent to

if(arr !is null) {...}

Because of how a null array is an empty array, some people expect the array
to be checked for whether it's non-empty in those situations, which can
cause confusion.

> 2. Should I prefer null or ""? I was hoping to return null to
> indicate "no string that match the criteria", and "some string"
> otherwise.

In most cases, it really doesn't matter in most situations whether you use
null or "" except that "" is automatically a string, whereas null can be
used as a literal for any type of dynamic array (in fact typeof(null) is its
own type in order to deal with that in generic code). The reason that

"" is null

is false is because all string literals in D have a null character one past
their end. This is so that you can pass them directly to C functions without
having to explicitly add the null character. e.g. both

printf("hello world");

and

printf("");

work correctly, because the compiler implicitly uses the ptr member of the
strings, and the C code happily reads past the end of the array to the null
character, whereas ""[0] would throw a RangeError in D code. Strings that
aren't literals don't have the null character unless you explicitly put it
there, and they require that you use ptr explicitly when calling C
functions, but for better or worse, string literals don't force that on you.

There are definitely experienced D programmers who differentiate between
null and empty arrays / strings in their code (in fact, that's why if(arr)
ultimately wasn't deprecated even though a number of people were pushing for
it because of how it confuses many people). However, there are also plenty
of D programmers who would argue that you should never treat null as special
with arrays because of how null arrays are empty instead of being treated as
their own thing.

Personally, I would say that if you want to differentiate between null and
empty, it can be done, but you need to be careful - especially if this is
going to be a function in a public API rather than something local to your
code. It's really easy to end up with a null array when you didn't expect to
- especially if your function is calling other functions that return arrays.

So, if you had a function that returned null when it fails, that _can_ work,
but you would either have to make sure that success never resulted in an
empty array being returned, or you would have to make it clear in the
documentation that 

Re: Cannot implicitly convert expression [[0, -1, 2], [4, 11, 2]] of type int[][] to const(int[2])[]

2020-01-31 Thread Jonathan M Davis via Digitalmars-d-learn
On Friday, January 31, 2020 5:43:44 AM MST MoonlightSentinel via 
Digitalmars-d-learn wrote:
> On Friday, 31 January 2020 at 12:37:43 UTC, Adnan wrote:
> > What's causing this?
>
> You mixed up the array lengths:
>
> const int[3][2] matA = [[0, -1, 2], [4, 11, 2]];
> const int[2][3] matB = [[3, -1], [1, 2], [6, 1]];
>
> matA is an SA containing <2> elements of type int[3].
> matB is an SA containing <3> elements of type int[2].

Specifically, the dimensions are read outwards from the variable name, so on
the left-hand side, that means that they go right-to-left, whereas on the
right-hand side, they go left-to-right. This is consistent with how it works
with types in C/C++ except that there, they put the dimensions for static
arrays on the right-hand side of the variable name, meaning that while you
have to read stuff like pointer types from left-to-right in C/C++, you don't
have to do that with static arrays. Ultimately, what D is doing is
consistent but confusing.

e.g. For C/C++

int** foo; // A pointer to a pointer to an int
int foo[5][2]; // A 5 dimensional array of two dimensional arrays of int
foo[4][1] = 7;

and for D:

int** foo; // A pointer to a pointer to an int
int[2][5] foo; // A 5 dimensional array of two dimensional arrays of int
foo[4][1] = 7;

For C/C++, you often don't realize how the rules work until you have to read
function pointers, because they put the static array lengths no the
right-hand side, and they actually allow you to put stuff like const in
multiple places instead of only in the place where it would be right
right-to-left. e.g. if the rule were followed strictly,

const int i = 0;

wouldn't be legal in C/C++. Rather, it would have to be

int const i = 0;

In reality, both work, but people end up using the first one. So,
ultimately, it adds to the confusion when dealing with more complex tyypes.
D doesn't have that problem, but since it used parens with type qualifiers,
it forces const to go on the left, making it less consistent. e.g.

const int i = 0;

or

const(int) i = 0;

So, neither C/C++ nor D is entirely consistent, but the basic rule is that
types are read outwards from the variable name, which is why you get the
weirdness with static array dimensions in D.

- Jonathan M Davis





Re: auto keyword

2020-01-30 Thread Jonathan M Davis via Digitalmars-d-learn
On Thursday, January 30, 2020 2:37:40 AM MST Michael via Digitalmars-d-learn 
wrote:
> auto is surely a nice feature. Nonetheless I'd prefer to use
> explicit types. So when reading a code and I see the auto keyword
> I also have to find out what kind of type is meant.
>
> I have a line of code that looks like this:
> auto elements = buf.to!string.strip.split(" ").filter!(a => a !=
> "");
>
> That line strips white space from buf, splits it, removes empty
> elements and returns an array of strings. At least I thought so.
>
> Indeed elements can be treated as a string slice, but if i
> replace auto by string[] the compiler complains:
> Error: cannot implicitly convert expression
> filter(split(strip(to(buf)), " ")) of type
> FilterResult!(__lambda1, string[]) to string[]
>
> In order to use an explicit type I wonder what kind of type I
> might use instead of auto?

For code like that, you don't. A lot of code in D - especially code that
uses ranges - uses what are called Voldemort types. They are declared inside
the function and you have no access to them. They follow a known API, so you
know what to do with them, but you they have no name that you have access
to. Realistically though, even if you had access to the type's name, you
wouldn't want to use it explicitly anyway, because usually, it's a templated
type which is instantiated with another templated type and is likely several
layers deep. Using the explicit names would be hideous. Years ago (before we
had Voldemort types), there was a bug in ddoc that made functions that
returned auto not show up in the documentation. So, all of std.algorithm
returned explicit types, and they were so hideous that it just scared
people. It works _far_ better to just understand how to use these types and
not use their names explicitly. It also makes the code far more resilient to
changes, since as long as the return type retains the same API, it doesn't
matter how the return type is changed. A function like filter could have
its return type changed to something else, and the code calling it wouldn't
care so long as it was the same kind of range.

Since you probably aren't familiar with ranges in D (or you wouldn't be
trying to use them by name), I suggest that you read this:

http://ddili.org/ders/d.en/ranges.html

And in addition to the fact that you pretty much have to use auto with
range-based code, you're pretty much going to have to get used to dealing
with D code using auto heavily, because that's what most D code does.
Certainly, sometimes, you can use type names explicitly, but it's common
practice to use auto most of the time. It can take a bit of getting used to,
but ultimately, it actually results in more maintainable code.

- Jonathan M Davis





Re: How to convert "string" to const(wchar)* ?

2020-01-29 Thread Jonathan M Davis via Digitalmars-d-learn
On Wednesday, January 29, 2020 12:16:29 AM MST Ferhat Kurtulmuş via 
Digitalmars-d-learn wrote:
> On Wednesday, 29 January 2020 at 06:53:15 UTC, Jonathan M Davis
>
> wrote:
> > On Tuesday, January 28, 2020 10:17:03 PM MST Marcone via
> >
> > Digitalmars-d-learn wrote:
> >> [...]
> >
> > Of course it is. string is immutable(char)[], and the
> > characters are in UTF-8. immutable(wchar)[] would would be
> > UTF-16. Even casting between those two types would result in
> > nonsense, because UTF-8 and UTF-16 are different encodings.
> > Casting between array or pointer types basically causes one
> > type to be interpreted as the other. It doesn't convert the
> > underlying data in any fashion. Also, strings aren't
> > null-terminated in D, so having a pointer to a random string
> > could result in a buffer overflow when you try to iterate
> > through the string via pointer as is typical in C code. D code
> > just uses the length property of the string.
> >
> > [...]
>
> + Just a reminder that string literals are null-terminated.

Yes, but unless you're using them directly, it doesn't really matter. Their
null character is one past their end and thus is not actually part of the
string itself as far as the type system is concerned. So, something as
simple as str ~ "foo" would mean that you weren't dealing with a
null-terminated string. You can do something like

printf("answer: %d\n", 42);

but if you mutate the string at all or create a new string from it, then
you're not dealing with a string with a null-terminator one past its end
anymore. Certainly, converting a string to wstring is not going to result in
the wstring being null-terminated without a null terminator being explicitly
appended to it.

Ultimately, that null-terminator one past the end of string literals is
pretty much just useful for being able to pass string literals directly to C
functions without having to explicitly put a null terminator on their end.

- Jonathan M Davis






Re: How to convert "string" to const(wchar)* ?

2020-01-28 Thread Jonathan M Davis via Digitalmars-d-learn
On Tuesday, January 28, 2020 10:17:03 PM MST Marcone via Digitalmars-d-learn 
wrote:
> How to convert "string" to const(wchar)* ?
> The code bellow is making confuse strange characters.
>
> cast(wchar*) str

Of course it is. string is immutable(char)[], and the characters are in
UTF-8. immutable(wchar)[] would would be UTF-16. Even casting between those
two types would result in nonsense, because UTF-8 and UTF-16 are different
encodings. Casting between array or pointer types basically causes one type
to be interpreted as the other. It doesn't convert the underlying data in
any fashion. Also, strings aren't null-terminated in D, so having a pointer
to a random string could result in a buffer overflow when you try to iterate
through the string via pointer as is typical in C code. D code just uses the
length property of the string.

I assume that with const(wchar)*, you want it to be a null-terminated string
of const(wchar). For that, what you basically need is a const(wchar)[] with
a null terminator, and then you need to get a pointer to its first
character. So, if you were to do that yourself, you'd end up with something
like

wstring wstr = to!wstring(str) ~ '\0';
const(wchar)* cwstr = wstr.ptr;

or more likely

auto = to!wstring(str) ~ '\0';
auto cwstr = wstr.ptr;

The function in the standard library for simplifying that is toUTF16z:

https://dlang.org/phobos/std_utf.html#toUTF16z

Then you can just do

auto cwstr = str.toUTF16z();

However, if you're doing this to pass a null-terminated string of UTF-16
characters to a C program (e.g. to the Windows API), be aware that if that
function stores that pointer anywhere, you will need to also store it in
your D code, because toUTF16z allocates a dynamic array to hold the string
that you're getting a pointer to, and if a C function holds on to that
pointer, the D GC won't see that it's doing that. And if the D GC doesn't
see any references to that array anywhere, it will likely collect that
memory. As long as you're passing it to a C function that just operates on
the memory and returns, it's not a problem, but it can definitely be a
problem if the C function stores that pointer even after the function has
returned. Keeping a pointer to that memory in your D code fixes that
problem, because then the D GC can see that that memory is still referenced
and thus should not be collected.

- Jonathan M Davis





Re: Lexicographic comparison of arrays (of chars)

2020-01-22 Thread Jonathan M Davis via Digitalmars-d-learn
On Wednesday, January 22, 2020 7:50:01 AM MST Per Nordlöw via Digitalmars-d-
learn wrote:
> On Wednesday, 22 January 2020 at 10:19:38 UTC, Jacob Carlborg
>
> wrote:
> > That looks like it's for internal use. There is a `compare`
> > method in the `TypeInfo` of each type.
>
> Will that incur an extra runtime cost compared to __cmp?

Regardless of the overhead involved, you really shouldn't be calling
functions that start with __ or any that are in an internal package. They're
not intended to be called directly by anything outside of druntime or
Phobos, and they could change at any time. In the case of
core.internal.array, the only reason that any of it is even exposed is
because it had to be when it was changed to a template.

- Jonathan M Davis






Re: range algorithms on container class

2020-01-18 Thread Jonathan M Davis via Digitalmars-d-learn
On Wednesday, January 15, 2020 9:13:05 AM MST Paul Backus via Digitalmars-d-
learn wrote:
> On Thursday, 9 January 2020 at 10:26:07 UTC, Jonathan M Davis
>
> wrote:
> > On Wednesday, January 8, 2020 10:56:20 PM MST rikki cattermole
> >
> > via Digitalmars-d-learn wrote:
> >> Slicing via the opSlice operator overload is a convention, not
> >> a requirement.
> >
> > It's not a requirement, but it's more than a convention. If you
> > use the container with foreach, the compiler will call opSlice
> > on the container to get a range. So, there's no need to
> > implement opApply to iterate over a container with foreach. You
> > could choose to implement a container with opApply and use a
> > function other than opSlice for getting a range, but the
> > compiler understands enough to try to slice the container for
> > you automatically when using it with foreach.
> >
> > - Jonathan M Davis
>
> Is this documented in the language spec anywhere? I don't see
> anything about it in the section on foreach [1] or the section on
> slice operator overloading [2].
>
> [1] https://dlang.org/spec/statement.html#foreach-statement
> [2] https://dlang.org/spec/operatoroverloading.html#slice

I don't know if it's in the spec or not, but IIRC, it was in TDPL. Either
way, I'm sure that it's been implemented. IIRC, there was even a bug related
to filter, because it had opSlice implemented on it for some reason when it
really shouldn't have, resulting in unexpected behavior when using the range
with foreach.

- Jonathan M Davis





Re: range algorithms on container class

2020-01-09 Thread Jonathan M Davis via Digitalmars-d-learn
On Wednesday, January 8, 2020 10:56:20 PM MST rikki cattermole via 
Digitalmars-d-learn wrote:
> Slicing via the opSlice operator overload is a convention, not a
> requirement.

It's not a requirement, but it's more than a convention. If you use the
container with foreach, the compiler will call opSlice on the container to
get a range. So, there's no need to implement opApply to iterate over a
container with foreach. You could choose to implement a container with
opApply and use a function other than opSlice for getting a range, but the
compiler understands enough to try to slice the container for you
automatically when using it with foreach.

- Jonathan M Davis





Re: range algorithms on container class

2020-01-09 Thread Jonathan M Davis via Digitalmars-d-learn
On Wednesday, January 8, 2020 10:28:23 PM MST Alex Burton via Digitalmars-d-
learn wrote:
> I am writing a specialised container class, and want to make it
> work like a normal array with range functions.
> After some struggle, I look at the code for std.container.array
> and see this comment :
>
> When using `Array` with range-based functions like those in
> `std.algorithm`,
>   * `Array` must be sliced to get a range (for example, use
> `array[].map!`
>   * instead of `array.map!`). The container itself is not a range.
>
>
> I had thought the compiler would make a generic random access
> range that works with the built in array for anything else with a
> simililar interface, but it doesn't.
>
> Does that mean it is not possible to have a library container
> implementation that works nicely with range algorithms ? Do you
> have to slice it like the comment suggests ?

You could make a container a range, but then iterating over it would pop
elements off of the container, which almost certainly isn't something that
you want. Containers are normally supposed to overload opSlice which returns
a range over the container so that you can iterate over the container
without altering it. And if you use a container with foreach, the compiler
will actually insert the opSlice call for you. Alternatively, you can define
opApply on the container. Outside of foreach though, you'd have to
explicitly slice the container.

- Jonathan M Davis





Re: @disable("reason")

2020-01-09 Thread Jonathan M Davis via Digitalmars-d-learn
On Wednesday, January 8, 2020 2:58:59 PM MST Marcel via Digitalmars-d-learn 
wrote:
> On Wednesday, 8 January 2020 at 07:03:26 UTC, Jonathan M Davis
>
> wrote:
> > On Tuesday, January 7, 2020 5:23:48 PM MST Marcel via
> >
> > Digitalmars-d-learn wrote:
> >> [...]
> >
> > In terms of an error message? Not really. You can put a
> > pragma(msg, ""); in there, but that would always print, not
> > just when someone tried to use it. I'd suggest that you just
> > put the information in the documentation. If they write code
> > that doesn't work because of you using
> >
> > [...]
>
> Oops, it appears I didn't actually understand this part of the
> language. I'm coming from C++ and I assumed both languages did
> this the same way, but thankfully I found a workaround that
> doesn't require doing what I wrote in the initial post and is
> better in general.

D took the approach of having all variables being default initalized with a
value that's known at compile-time so that you don't risk dealing with
garbage values as can happen in C/C++. For structs, the result is that
default constructors aren't a thing, because the default vaule has to be a
fixed value known at compile time, and that doesn't work with default
constructors, which can run arbitrary code at runtime. And with non-default
constructors, a struct is actually initialized with its default value before
its constructor is even run. So, you don't risk reading garbage values in
the object's member variables before properly initializing them.

There are times when the lack of default constructors can be annoying, but
it avoids a class of problems that C++ has, and overall, it works quite
well.

The ability to @disable this(); was added later in order to facilitate rarer
cases where using the default value for a type would be problematic (e.g.
when a type absolutely needed to have some code run at runtime when
constructing it for it to work properly). However, because the language was
designed around the idea that all objects would be default-initialized,
disabling default initialization tends to make some typical stuff not work
with that type (like out parameters or arrays, since they both require the
init value).

Ultimately, D's approach is a tradeoff. Some of the problems that you can
have in C++ are eliminated, but it introduces some complications that don't
exist in C++.

> I also agree with Adam, it can make for a nice little feature in
> D.

I doubt that there would be much resistance to it, but since you almost
certainly need to read the documentation to understand how to use the type
properly if it can't be default-initialized, I don't know that it would
actually add much benefit in practice. Either way, it's not something that's
come up often enough for anyone to push for such a language change.

- Jonathan M Davis





Re: @disable("reason")

2020-01-08 Thread Jonathan M Davis via Digitalmars-d-learn
On Wednesday, January 8, 2020 4:54:06 AM MST Simen Kjærås via Digitalmars-d-
learn wrote:
> On Wednesday, 8 January 2020 at 07:03:26 UTC, Jonathan M Davis
>
> wrote:
> > you could just document that no one should ever use its init
> > value explicitly, and that they will have bugs if they do
>
> You also create a static init member marked @disable:
>
> struct S {
>  @disable this();
>  @disable static S init();
> }
>
> This will give sensible error messages anywhere .init is being
> used. Now, Phobos and other libraries might expect that .init is
> always working, so this could potentially be a problem.

That's likely to break a _lot_ of generic code, because init is used heavily
in template constraints and static if conditions as the way to get a value
of that type to test. It's also actually been argued before that it should
be illegal to declare a symbol called init on any type, which is one reason
why the init member was removed from TypeInfo. I'd strongly advise against
anyone declaring a struct or class member named init whether it's @disabled
or not.

- Jonathan M Davis






Re: @disable("reason")

2020-01-07 Thread Jonathan M Davis via Digitalmars-d-learn
On Tuesday, January 7, 2020 5:23:48 PM MST Marcel via Digitalmars-d-learn 
wrote:
> Hello!
> I'm writing a library where under certain conditions i need all
> the default constructors to be disabled. I would like to tell the
> user why they can't instantiate the struct.
> Is there a way to do that?

In terms of an error message? Not really. You can put a pragma(msg, ""); in
there, but that would always print, not just when someone tried to use it.
I'd suggest that you just put the information in the documentation. If they
write code that doesn't work because of you using

@disable this();

the documentation is where people will look to find out why anyway.

And BTW, structs don't have default constructors in D. So, maybe it's just a
question of terminology rather than you misunderstanding the semantics, but
all variables in D get default initialized with their type's init value,
including structs, and you can't actually declare a default constructor for
a struct. So, if you have

@disable this();

in a struct, all that that's doing is disabling default initialization, not
default construction. So, code like

MyStruct ms;

or

MyStruct[] arr;

won't compile, because they require default initialization. But even then,
you don't actually get rid of the default value of the struct. You just make
it so that it doesn't get used implicitly. Code such as

auto ms = MyStruct.init;

or

MyStruct ms = MyStruct.init;

will still work. So, you still potentially have to worry about the type's
init value being used (though you could just document that no one should
ever use its init value explicitly, and that they will have bugs if they do,
not that that actually prevents people from using it incorrectly; it just
informs them that they shouldn't). It's unlikely that many people will try
to use the init value explicitly, but some generic code may do so (e.g.
IIRC, std.algorithm's move function uses it on the source variable after
moving the value to the target variable).

- Jonathan M Davis





Re: @safe std.file.read

2020-01-06 Thread Jonathan M Davis via Digitalmars-d-learn
On Monday, January 6, 2020 8:52:01 AM MST Steven Schveighoffer via 
Digitalmars-d-learn wrote:
> On 1/6/20 5:07 AM, WebFreak001 wrote:
> > I was wondering, how are you supposed to use std.file : read in @safe
> > code when it returns a void[] but you want to get all bytes in the file?
> >
> > Is void[] really the correct type it should be returning instead of
> > ubyte[] when it just reads a (binary) file to memory? Or should void[]
> > actually be castable to ubyte[] in @safe code?
>
> I feel like this conversation has been had before. But I think it should
> be ubyte[]. Not sure why it's void[]. Perhaps for symmetry with write,
> which takes void[] (for good reason)?

I think that in previous discussions, it was decided that in general, when
you're dealing with something like reading from / write to a file or a
socket, writing should accept void[], because then you can write any binary
data to it without casting (including objects which are being serialized),
whereas reading should give you ubyte[] or const(ubyte)[], because what
you're getting from the OS is bytes of data, and it's up to the program to
figure out what to do with them.

- Jonathan M Davis





Re: Thin UTF8 string wrapper

2019-12-07 Thread Jonathan M Davis via Digitalmars-d-learn
On Saturday, December 7, 2019 5:23:30 AM MST Joseph Rushton Wakeling via 
Digitalmars-d-learn wrote:
> On Saturday, 7 December 2019 at 03:23:00 UTC, Jonathan M Davis
>
> wrote:
> > The module to look at here is std.utf, not std.encoding.
>
> Hmmm, docs may need updating then -- several functions in
> `std.encoding` explicitly state they are replacements for
> `std.utf`.  Did you mean `std.uni`?

> It is honestly a bit confusing which of these 3 modules to use,
> especially as they each offer different (and useful) tools.  For
> example, `std.utf.validate` is less useful than
> `std.encoding.isValid`, because it throws rather than returning a
> bool and giving the user the choice of behaviour.  `std.uni`
> doesn't seem to have any equivalent for either.
>
> Thanks in any case for the as-ever characteristically detailed
> and useful advice :-)

There may have been some tweaks to std.encoding here and there, but for the
most part, it's pretty ancient. Looking at the history, it's Seb who marked
some if it as being a replacement for std.utf, which is just plain wrong.
Phobos in general uses std.utf for dealing with UTF-8, UTF-16, and UTF-32,
not std.encoding. std.encoding is an old module that's had some tweaks done
to it but which probably needs a pretty serious overhaul. The only thing
that I've ever use it for is BOM stuff.

std.utf.validate does need a replacement, but doing so gets pretty
complicated. And looking at std.encoding.isValid, I'm not sure that what it
does is any better from simply wrapping std.utf.validate and returning a
bool based on whether an exception was thrown. Depending on the string, it
would actually be faster to use validate, because std.encoding.isValid
iterates through the entire string regardless. The way it checks validity is
also completely different from what std.utf does. Either way, some of the
std.encoding internals do seem to be an alternate implementation of what
std.utf has, but outside of std.encoding itself, std.utf is what Phobos uses
for UTF-8, UTF-16, and UTF-32, not std.encoding.

I did do a PR at one point to add isValidUTF to std.utf so that we could
replace std.utf.validate, but Andrei didn't like the implementation, so it
didn't get merged, and I haven't gotten around to figuring out how to
implement it more cleanly.

- Jonathan M Davis





Re: Thin UTF8 string wrapper

2019-12-06 Thread Jonathan M Davis via Digitalmars-d-learn
On Friday, December 6, 2019 9:48:21 AM MST Joseph Rushton Wakeling via 
Digitalmars-d-learn wrote:
> Hello folks,
>
> I have a use-case that involves wanting to create a thin struct
> wrapper of underlying string data (the idea is to have a type
> that guarantees that the string has certain desirable properties).
>
> The string is required to be valid UTF-8.  The question is what
> the most useful API is to expose from the wrapper: a sliceable
> random-access range?  A getter plus `alias this` to just treat it
> like a normal string from the reader's point of view?
>
> One factor that I'm not sure how to address w.r.t. a full range
> API is how to handle iterating over elements: presumably they
> should be iterated over as `dchar`, but how to implement a
> `front` given that `std.encoding` gives no way to decode the
> initial element of the string that doesn't also pop it off the
> front?
>
> I'm also slightly disturbed to see that `std.encoding.codePoints`
> requires `immutable(char)[]` input: surely it should operate on
> any range of `char`?
>
> I'm inclining towards the "getter + `alias this`" approach, but I
> thought I'd throw the problem out here to see if anyone has any
> good experience and/or advice.
>
> Thanks in advance for any thoughts!

The module to look at here is std.utf, not std.encoding. decode and
decodeFront can be used to get a code point if that's what you want, whereas
byCodeUnit and byUTF can be used to get a range over code units or code
points. There's also byCodePoint and byGrapheme in std.uni. std.encoding is
old and arguably needs an overhaul. I don't think that I've ever done
anything with it other than for dealing with BOMs.

If you provide a range of UTF-8 code units, then it will just work with any
code that's written to work with a range of any character type, whereas if
you specifically need to have it be a range of code points or graphemes,
then using the wrappers from std.utf or std.uni will get you that. And there
really isn't any reason to restrict the operations on a range of char the
way that std.range.primitives does for string. If you're dealing with a
function that was specifically written to operate on any range of
characters, then it's unnecessary, and if it's just a normal range-based
function which isn't specialized for ranges of characters, then it's going
to iterate over whatever the element type of the range is. So, you'll need
to use a wrapper like byUTF, byCodePoint, or byGrapheme to get whatever the
correct behavior is depending on what you're trying to do.

The main hiccup is that a lot of Phobos is basically written with the idea
that ranges of characters will be ranges of dchar. Some of Phobos has been
fixed so that it doesn't, but plenty of it hasn't been. However, what that
usually means is that the code just operates on the element type and
special-cases for narrow strings, or it's specifically written to operate on
ranges of dchar. For cases like that, byUTF!dchar or byCodePoint will likely
work; alternatively, you can provide a way to access the underlying string
and just have them operate directly on the string, but depending on what
you're trying to do with your wrapper, exposing the underlying string may or
may not be a problem (given that string has immutable elements though, it's
probably fine so long as you don't provide a reference to the string
itself).

In general, I'd strongly advise against using alias this with range-based
code (or really, generic code in general). Depending, it _can_ work, but
it's also an easy source of bugs. Unless the code forces the conversion,
what you can easily get is some of the code operating directly on the type
and some of it doing the implicit conversion to operate on the type. Best
case, that results in compilation errors, but it could also result in subtle
bugs. It's far less error-prone to require that the conversion be done
explicitly.

So, if all you're really trying to do is provide some guarantees about how
the string was constructed but then are looking to essentially just have it
be a string after that, it would probably be simplest to make it so that
your wrapper type doesn't have much in the way of operations and that it
just provides a property to access the underlying string. Then the type
itself isn't a range, and any code that wants to operate on the data can
just use the property to get the underlying string and use it as a string
after that. That approach basically completely sidesteps the issue of how to
treat the data as a range, since you get the normal behavior for strings for
any code that does much more than just pass around the data. You _do_ lose
the knowledge that the wrapper type gave you about the state of the string
once you start actually operating on the data, but once you start operating
on it, that knowledge is probably no longer valid anyway (especially if
you're passing it to a function which is going to return a wrapper range to
mutate the elements in the range 

Re: opCmp with and without const

2019-12-06 Thread Jonathan M Davis via Digitalmars-d-learn
On Friday, December 6, 2019 12:03:45 AM MST berni44 via Digitalmars-d-learn 
wrote:
> In std.typecons, in Tuple there are two opCmp functions, that are
> almost identical; they only differ by one being const and the
> other not:
>
>  int opCmp(R)(R rhs)
>  if (areCompatibleTuples!(typeof(this), R, "<"))
>  {
>  static foreach (i; 0 .. Types.length)
>  {
>  if (field[i] != rhs.field[i])
>  {
>  return field[i] < rhs.field[i] ? -1 : 1;
>  }
>  }
>  return 0;
>  }
>
>  int opCmp(R)(R rhs) const
>  if (areCompatibleTuples!(typeof(this), R, "<"))
>  {
>  static foreach (i; 0 .. Types.length)
>  {
>  if (field[i] != rhs.field[i])
>  {
>  return field[i] < rhs.field[i] ? -1 : 1;
>  }
>  }
>  return 0;
>  }
>
>
> What is the reason for having this? (I guess, that it's because
> the function may indirectly call opCmp of other types which may
> or may not be const.)
>
> My real question is: Can this code duplication be avoided
> somehow? (I ask, because I've got a PR running, which increases
> the size of these functions and it doesn't feel good to have two
> long, almost identical functions.)

The issue is that there's no guarantee that the types being wrapped have a
const opCmp. So, you can't just slap const or inout on Tuple's opCmp and
have it work, but you do want it to be const if it can be const. So, two
overloads are declared, and the template constraint takes care of checking
whether that particular overload can be instantiated.

A mixin could be used for the function bodies to avoid duplicating the
internals, and it may be possible to use template this parameters as Paul
Backus suggested (I'm not very familiar with template this parameters, so I
don't know how well they'll work in this particular case), but ultimately,
one way or another, you need to have a non-const opCmp declared for when the
wrapped types don't have an opCmp that works with const and a const or inout
opCmp for when the wrapped types do have an opCmp that works with const.

- Jonathan M Davis





Re: Unexpectedly nice case of auto return type

2019-12-03 Thread Jonathan M Davis via Digitalmars-d-learn
On Tuesday, December 3, 2019 3:23:20 AM MST Basile B. via Digitalmars-d-
learn wrote:
> On Tuesday, 3 December 2019 at 10:19:02 UTC, Jonathan M Davis
>
> wrote:
> > On Tuesday, December 3, 2019 3:03:22 AM MST Basile B. via
> >
> > Digitalmars-d- learn wrote:
> >> [...]
> >
> > There isn't much point in giving the type of null an explicit
> > name given that it doesn't come up very often, and typeof(null)
> > is quite explicit about what the type is. Also, anyone doing
> > much generic programming in D is going to be well versed in
> > typeof. They might not know about typeof(null) explicitly, but
> > they should recognize what it means when they see it, and if
> > someone were trying to get the type of null, it would be the
> > obvious thing to try anyway. And typeof(null) isn't even the
> > prime case where typeof gets used on something other than an
> > object. From what I've seen, typeof(return) gets used far more.
> >
> > [...]
>
> you're right but I see two cases:
>
> - transpiling
> - header generation

I don't see why either of those would be a problem. For a transpiler,
typeof(null) and an explicit type name for typeof(null) would be the same
thing, and it would have to be translated to something that would work in
the other language either way. As for header generation, do you mean .di
files? They'd just use typeof(null) explicitly, whereas if you're talking
about something like having extern(C) functions in D and declaring a
corresponding .h file, typeof(null) wouldn't work anyway, because there is
no equivalent in C. In either case, whether you represent the type as
typeof(null) or by an explicit name in the D source code is irrelevant. It
would mean the same thing regardless. Having an explicit name wouldn't
really be any different from declaring an alias like

alias TypeOfNull = typeof(null);

The type is the same either way and would be treated the same by the
compiler or by any tool that needed to translate it to another language.
It's what the type is that matters, not how its represented in the D source
code. The only real difference would be what the programmer would see when
interacting with the D source code. It would be like arguing over whether
the root class object should be called Object or Root. It would be the same
thing either way, just with a different name.

- Jonathan M Davis





Re: Unexpectedly nice case of auto return type

2019-12-03 Thread Jonathan M Davis via Digitalmars-d-learn
On Tuesday, December 3, 2019 3:03:22 AM MST Basile B. via Digitalmars-d-
learn wrote:
> On Tuesday, 3 December 2019 at 09:58:36 UTC, Jonathan M Davis
>
> wrote:
> > On Tuesday, December 3, 2019 12:12:18 AM MST Basile B. via
> >
> > Digitalmars-d- learn wrote:
> >> I wish something like this was possible, until I change the
> >> return type of `alwaysReturnNull` from `void*` to `auto`.
> >>
> >>
> >> ---
> >> class A {}
> >> class B {}
> >>
> >> auto alwaysReturnNull() // void*, don't compile
> >> {
> >>
> >>  writeln();
> >>  return null;
> >>
> >> }
> >>
> >> A testA()
> >> {
> >>
> >>  return alwaysReturnNull();
> >>
> >> }
> >>
> >> B testB()
> >> {
> >>
> >>  return alwaysReturnNull();
> >>
> >> }
> >>
> >> void main()
> >> {
> >>
> >>  assert( testA() is null );
> >>  assert( testB() is null );
> >>
> >> }
> >> ---
> >>
> >> OMG, isn't it nice that this works ?
> >>
> >> I think that this illustrates an non intuitive behavior of auto
> >> return types.
> >> One would rather expect auto to work depending on the inner
> >> return type.
> >
> > The void* version doesn't work, because void* doesn't
> > implicitly convert to a class type. It has nothing to do with
> > null. auto works thanks to the fact that typeof(null) was added
> > to the language a while back, and since class references can be
> > null, typeof(null) implicitly converts to the class type.
> > Before typeof(null) was added to the language, null by itself
> > had no type, since it's just a literal representing the null
> > value for any pointer or class reference. The result was that
> > using null in generic code or with auto could run into issues.
> > typeof(null) was added to solve those problems.
> >
> > - Jonathan M Davis
>
> That's interesting details of D developement. Since you reply to
> the first message I think you have not followed but in the last
> reply I told that maybe we should be able to name the type of
> null. I think this relates to TBottom too a bit.

There isn't much point in giving the type of null an explicit name given
that it doesn't come up very often, and typeof(null) is quite explicit about
what the type is. Also, anyone doing much generic programming in D is going
to be well versed in typeof. They might not know about typeof(null)
explicitly, but they should recognize what it means when they see it, and if
someone were trying to get the type of null, it would be the obvious thing
to try anyway. And typeof(null) isn't even the prime case where typeof gets
used on something other than an object. From what I've seen, typeof(return)
gets used far more.

As for TBottom, while the DIP does give it a relationship to null, they're
still separate things, and giving typeof(null) a name wouldn't affect
TBottom at all.

- Jonathan M Davis





Re: Unexpectedly nice case of auto return type

2019-12-03 Thread Jonathan M Davis via Digitalmars-d-learn
On Tuesday, December 3, 2019 12:12:18 AM MST Basile B. via Digitalmars-d-
learn wrote:
> I wish something like this was possible, until I change the
> return type of `alwaysReturnNull` from `void*` to `auto`.
>
>
> ---
> class A {}
> class B {}
>
> auto alwaysReturnNull() // void*, don't compile
> {
>  writeln();
>  return null;
> }
>
> A testA()
> {
>  return alwaysReturnNull();
> }
>
> B testB()
> {
>  return alwaysReturnNull();
> }
>
> void main()
> {
>  assert( testA() is null );
>  assert( testB() is null );
> }
> ---
>
> OMG, isn't it nice that this works ?
>
> I think that this illustrates an non intuitive behavior of auto
> return types.
> One would rather expect auto to work depending on the inner
> return type.

The void* version doesn't work, because void* doesn't implicitly convert to
a class type. It has nothing to do with null. auto works thanks to the fact
that typeof(null) was added to the language a while back, and since class
references can be null, typeof(null) implicitly converts to the class type.
Before typeof(null) was added to the language, null by itself had no type,
since it's just a literal representing the null value for any pointer or
class reference. The result was that using null in generic code or with auto
could run into issues. typeof(null) was added to solve those problems.

- Jonathan M Davis





Re: Building and running DMD tests

2019-12-01 Thread Jonathan M Davis via Digitalmars-d-learn
On Sunday, December 1, 2019 8:20:42 AM MST Per Nordlöw via Digitalmars-d-
learn wrote:
> Is it possible to compile and run unittest of dmd without
> druntime and phobos?
>
> If so, how?
>
> I'm trying the following under dmd root:
>
>  make -C src -f posix.mak unittest
>  ./generated/linux/release/64/dmd-unittest
>
> but that doesn't compile my file of interest
>
> test/compilable/traits.d
>
> .
>
> How can I make sure that all the files under /test/compilable
> compiles?

dmd's tests are designed to be run after you've built druntime and Phobos.
If you look at the tests, many of them import modules from both druntime
and/or Phobos. IIRC, there has been some discussion about whether that
should be changed, but AFAIK, there has been no agreement to do so. I'm also
not sure that it even _can_ be done with regards to druntime, because every
D program that isn't compiled with -betterC requires at least druntime. So,
at most, it would probably mean not requiring Phobos, but either way, at the
moment, the tests in general expect Phobos to have been built and be
available. I don't know how possible it is to get around that with a
specific test module that doesn't actually use Phobos, but that's not how
the tests are normally run, and you'd need druntime regardless.

In addition, I believe that the unittest target that you're trying to build
is specifically for running all of the unittest blocks in the dmd source
code, not for running the tests in the test folder. Those are built using
the makefile in the test folder or by running the test target from the
primary makefile with the target test (which also runs the unittest blocks
in the src folder), whereas you're specifically using the makefile in src.

- Jonathan M Davis






Re: How to create DDoc for string mixin generated functions?

2019-11-26 Thread Jonathan M Davis via Digitalmars-d-learn
On Monday, November 25, 2019 9:25:08 AM MST ParticlePeter via Digitalmars-d-
learn wrote:
> I am producing a bunch of functions/methods through string
> mixins. I also generated DDoc comments for those functions, in
> the hope that they would produce proper documentation, but they
> don't. So how can this be accomplished?

Right now, you don't. The ddoc and function signature have to be directly in
the source code for them to be seen for ddoc generation. The closest I'm
aware of to being able to do anything along the lines of mixing in
documentation is to use template mixins, and IIRC, you not only have to have
ddoc on the mixed in symbols, but you need to put at least an empty ddoc
comment on the statement that mixes in the template. e.g. std.exception has

/++
...
  +/
mixin template basicExceptionCtors()
{
/++
Params:
msg  = The message for the exception.
file = The file where the exception occurred.
line = The line number where the exception occurred.
next = The previous exception in the chain of exceptions, if 
any.
+/
this(string msg, string file = __FILE__, size_t line = __LINE__,
 Throwable next = null) @nogc @safe pure nothrow
{
super(msg, file, line, next);
}

/++
Params:
msg  = The message for the exception.
next = The previous exception in the chain of exceptions.
file = The file where the exception occurred.
line = The line number where the exception occurred.
+/
this(string msg, Throwable next, string file = __FILE__,
 size_t line = __LINE__) @nogc @safe pure nothrow
{
super(msg, file, line, next);
}
}

and to have those constructors show up in the documentation when mixed in,
you have to do something like:

/++
My exception class.
  +/
class MyException : Exception
{
///
mixin basicExceptionCtors;
}

Without the empty ddoc comment, the documentation on the mixed in symbols
does not show up, but either way, nothing like that can currently be done
with string mixins.

There's on open bug report / enhancement request somewhere on bugzilla to
fix it so that you can document string mixins, but unless someone has done
something to fix that very recently, no one has yet to implement a fix.

- Jonathan M Davis





Re: static assert(version(x)) ?

2019-11-26 Thread Jonathan M Davis via Digitalmars-d-learn
On Tuesday, November 26, 2019 4:29:18 AM MST S.G via Digitalmars-d-learn 
wrote:
> On Tuesday, 26 November 2019 at 10:24:00 UTC, Robert M. Münch
>
> wrote:
> > How can I write something like this to check if any of a set of
> > specific versions is used?
> >
> > static assert(!(version(a) | version(b) | version(c)):
> >
> > The problem is that I can use version(a) like a test, and the
> > symbol a is not accessbile from assert (different,
> > non-accessible namespace).
>
> BTW D language designers are against boolean eval of version.
> It's not a technical restriction, it's just that they don't want
> this to work.

Basically, Walter considers it to be a prime source of bugs in C/C++ code.
druntime, Phobos, etc. consistently do stuff like

version(Posix)
{
}
else version(Windows)
{
}
else
static assert(false, "platform unsupported);

when it needs code to differ depending on platform or architecture or
whatever. And if that means duplicating some code in each version block,
then it means duplicating some code in each version block. static if can be
used instead of version blocks to get boolean conditions, and local version
identifiers can be defined which combine some set of version identifiers,
but such practices are discouraged for D programmers in general, and they're
basically forbidden in official source code. The only case I'm aware of
where anything like that is used in druntime or Phobos is for darwin stuff,
since darwin isn't a predefined identifier.

- Jonathan M Davis






Re: What is the point of a synchronized lock on a single return statement?

2019-11-25 Thread Jonathan M Davis via Digitalmars-d-learn
On Monday, November 25, 2019 1:22:17 AM MST Andrej Mitrovic via Digitalmars-
d-learn wrote:
> From:
> https://github.com/dlang/phobos/blob/10b9174ddcadac52f6a1ea532deab3310d3a8
> c03/std/concurrency.d#L1913-L1916:
>
> -
> ///
> final @property bool isClosed() @safe @nogc pure
> {
>  synchronized (m_lock)
>  {
>  return m_closed;
>  }
> }
> -
>
> I don't understand the purpose of this lock. The lock will be
> released as soon as the function returns, and it returns a copy
> of a boolean anyway. Am I missing something here?

It ensures that no other code that locks m_lock is running when m_closed is
accessed. I'd have to study std.concurrency in detail to know for sure why
that would be needed, but it's not atypical when trying to maintain
consistent state when multiple threads are interacting with each other.

- Jonathan M Davis





Re: why local variables cannot be ref?

2019-11-25 Thread Jonathan M Davis via Digitalmars-d-learn
On Monday, November 25, 2019 1:32:53 AM MST H. S. Teoh via Digitalmars-d-
learn wrote:
> On Mon, Nov 25, 2019 at 08:07:50AM +, Fanda Vacek via
> Digitalmars-d-learn wrote: [...]
>
> > But anyway, pointers are not allowed in @safe code, so this is not
> > always solution.
>
> [...]
>
> This is incorrect. Pointers *are* allowed in @safe code. Pointer
> *arithmetic* is not allowed.

That and taking the address of a local variable is @system, though with
-dip1000, it becomes @safe by making the result scope, which restricts what
you can do with it. And of course, @trusted can be used where appropriate to
make it so that @system code can be used from @safe code, though obviously,
that means that it's up to the programmer to make sure that they verify that
what the @trusted code is doing is actually @safe.

Further, with regards to taking the address of a local variable being
@system, ref would be @system for exactly the same reasons save for the fact
that there are various restrictions in place to ensure that it can't be used
in a manner which would allow the underlying pointer to exist longer than
the address that it refers to. Allowing variables in general to be declared
as ref instead of only allowing it in restricted circumstances such as
function parameters and return types would put ref in the same @system
quagmire that exists with regards to taking the address of a local variable.
By restricting ref, that problem is avoided, whereas pointers still allow
for full freedom but in return, they require that certain operations that
relate to them be @system (like &, ++, and --).

- Jonathan M Davis





Re: Parsing with dxml

2019-11-18 Thread Jonathan M Davis via Digitalmars-d-learn
On Sunday, November 17, 2019 11:44:43 PM MST Joel via Digitalmars-d-learn 
wrote:
> I can only parse one row successfully. I tried increasing the
> popFronts, till it said I'd gone off the end.
>
> Running ./app
> core.exception.AssertError@../../../../.dub/packages/dxml-0.4.1/dxml/sourc
> e/dxml/parser.d(1457): text cannot be called with elementEnd
> 
> ??:? _d_assert_msg [0x104b3981a]
> ../../JMiscLib/source/jmisc/base.d:161 pure @property @safe
> immutable(char)[] dxml.parser.EntityRange!(dxml.parser.Config(1,
> 1, 1, 1), immutable(char)[]).EntityRange.Entity.text()
> [0x104b2297b]
> source/app.d:26 _Dmain [0x104aeb46e]
> Program exited with code 1
>
> ```
> 
>
> http://www.w3.org/2001/XMLSchema-instance;>
>
>   01001001
>   1
>   1
>   1
>   In the beginning God created the heavens and the
> earth.
>
>
>
>   01001002
>   1
>   1
>   2
>   And the earth was waste and void; and darkness
> was upon the face of the deep: and the Spirit of God moved upon
> the face of the waters.
>
>
> ```
>
> ```d
> void main() {
>  import std.stdio;
>  import std.file : readText;
>  import dxml.parser;
>  import std.conv : to;
>
>  struct Verse {
>  string id;
>  int b, c, v;
>  string t;
>  }
>
>  auto range = parseXML!simpleXML(readText("xmltest.xml"));
>
>  // simpleXML skips comments
>
>  void pops(int c) {
>  foreach(_; 0 .. c)
>  range.popFront();
>  }
>  pops(3);
>
>  Verse[] vers;
>  foreach(_; 0 .. 2) {
>  Verse ver;
>  ver.id = range.front.text;
>  pops(3);
>  ver.b = range.front.text.to!int;
>  pops(3);
>  ver.c = range.front.text.to!int;
>  pops(3);
>  ver.v = range.front.text.to!int;
>  pops(3);
>  ver.t = range.front.text;
>
>  with(ver)
>  vers ~= Verse(id,b,c,v,t);
>
>  pops(2);
>  }
>  foreach(verse; vers) with(verse)
>  writeln(id, " Book: ", b, " ", c, ":", v, " -> ", t);
> }
> ```

You need to be checking the type of the entity before you call either name
or text on it, because not all entities have a name, and not all entities
have text - e.g.  is an EntityType.elementStart, so it has
a name (which is "field"), but it doesn't have text, whereas the 01001001
between the  and  tags has no name but does have
text, because it's an EntityType.text. If you call name or text without
verifying the type first, then you're almost certainly going to get an
assertion failure at some point (assuming that you don't compile with
-release anyway), since you're bound to end up with an entity that you don't
expect at some point (either because you were wrong about where you were in
the document, or because the document didn't match the layout that was
expected).

Per the assertion's message, you managed to call text on an
EntityType.elementEnd, and per the stack trace, text was called on this line

 ver.id = range.front.text;

If I add

 if(range.front.type == EntityType.elementEnd)
 {
 writeln(range.front.name);
 writeln(range.front.pos);
 }

right above that, I get

row
TextPos(11, 4)

indicating that the end tag was  and that it was on line 11, 4 code
units in (and since this is ASCII, that would be 4 characters). So, you
managed to parse all of the *** lines but didn't correctly
deal with the end of that section.

If I add

writeln(range.front);

right before

pops(2);

then I get:

Entity(text, TextPos(10, 25), , Text!(ByCodeUnitImpl)(In the beginning God
created the heavens and the earth., TextPos(10, 25)))

So, prior to popping twice, it's on the text between  and
, which looks like it's what you intended. If you look at the XML
after that, it should be clear why you're in the wrong place afterwards.

Since at that point, range.front is on the EntityType.text between
 and , popping once makes it so that range.front is
. And popping a second time makes range.front , which is where
the range is when it the tries to call text at the top of the loop.
Presumably, you want it to be on the EntityType.text in

01001002

To get there from , you'd have to pop once to get to , a second
time to get to , and a third time to get to 01001002. So, if you had

pops(5);

instead of

pops(2);

the range would be at the correct place at the top of the loop - though it
would then be the wrong number of times to pop the second time around. With
the text as provided, it would throw an XMLParsingException when it reached
the end of the loop the second time, because the XML document doesn't have
the matching  tag, and with that fixed, you end up with an
assertion failure, because popFront was called on an empty range (since
there aren't 7 elements left in the range at that point):

core.exception.AssertError@../../.dub/packages/dxml-0.4.0/dxml/source/dxml
/parser.d(1746): It's illegal to call 

Re: Should I stop being interested in D language if I don't like to see template instantiation in my code?

2019-11-13 Thread Jonathan M Davis via Digitalmars-d-learn
On Wednesday, November 13, 2019 7:01:13 AM MST BoQsc via Digitalmars-d-learn 
wrote:
> I don't like to see exclamation marks in my code in as weird
>
> syntax as these ones:
> > to!ushort(args[1])
> > s.formattedRead!"%s!%s:%s"(a, b, c);
>
> I'm not sure why, but template instantiation syntax is prevalent
> in the documentation examples of d lang libraries. It almost
> seems like every other example contains at least one or two  of
> them.
>
> It look horrible, and I'm feeling like I'm being forced/coerced
> to learn from examples that do not provide alternatives to the
> template instantiation syntax. Even if the alternative examples
> were provided, why would anyone want to have syntax as ugly and
> weird as current template instantiation syntax with exclamation
> point in the middle of the statement with all other things that
> come with it.

D uses !() rather than <>, because it's not ambiguous, allowing for the
parser to be context-free (whereas a language like C++ or Java has to
actually process the context in order to know whether something like the >>
in vector> is a template instantation or a shift
operation; parsing is _much_ cleaner if it can be context-free). D then
allows the parens to be dropped when there's only one template argument,
which is why you get stuff like s.formattedRead!"" instead of
s.formattedRead!("").

Regardless of the syntax though, like C++'s standard library, D's standard
library uses templates quite a bit, and it's extremely common for D code in
general to use templates. I don't know why you think that using an
exclamation point for template instantiations is ugly, but if you can't
stand it, you're not going to be happy with D, because you're going to see
it quite a lot in typical D code.

- Jonathan M Davis





Re: Alias sleep(int) for Thread.sleep(dur!("seconds")( int ));

2019-11-12 Thread Jonathan M Davis via Digitalmars-d-learn
On Tuesday, November 12, 2019 2:24:54 PM MST Marcone via Digitalmars-d-learn 
wrote:
> I am using this function to sleep, but I want a simple Alias. How
> can I alias this?
>
> // Function sleep(int)
> void sleep(int seconds){
>   Thread.sleep(dur!("seconds")( seconds ));
> }
>
> sleep(1); // Using function.

An alias just gives a different name for a symbol. It can't pass arguments
for you or call a function and pass its result to another. So, while you
could create an alias for Thread.sleep, you'd have to call it exactly like
you'd call Thread.sleep - just with a different name. If you want to do
something like have it accept an int instead of a Duration, then you need a
wrapper function like you're already doing.

Now, in core.time, dur!"seconds" has an alias named seconds, so if you're
simply looking to reduce how much typing you're doing, you could have
Thread.sleep(seconds(42)), or if you aliased Thread.sleep to sleep, you
could do sleep(seconds(42)), but you can't do something like sleep(42)
without a wrapper function.

In any case, I'd suggest that you avoid passing around naked integers as
time values. Thread.sleep takes a Duration specifically because it makes for
clearer code and is less error-prone than using a naked integer (since a
naked integer has no units).

- Jonathan M Davis





Re: Unexpected aliasing

2019-11-11 Thread Jonathan M Davis via Digitalmars-d-learn
On Monday, November 11, 2019 12:17:37 PM MST Bastiaan Veelo via Digitalmars-
d-learn wrote:
> Recently I got my first surprise with our use of D. The symptom
> was that two local variables in two different functions appeared
> to be sharing data.
>
> A simplified example is shown below (the original was machine
> translated from Pascal and involved templates and various levels
> of indirection). What I did not know is that the initial value of
> struct members is a compile time feature, apparently. What I
> suspect is happening is that the array lives in the static data
> segment (or is created in the module constructor?) and that the
> slices inside arr1 and arr2 get initialised to point to that same
> array.
>
> I could use some help in rewriting the code below so that arr1
> and arr2 each have their own data; ideally with minimal changes
> so that I can make the transcompiler do the right thing.
>
> Thanks!
> Bastiaan.
>
> void main()
> {
>   import std.stdio;
>
>   WrapIntegerArray arr1;
>   arr1[0] = 42;
>
>   WrapIntegerArray arr2;
>
>   writeln(arr2[0]); // 42, not 0.
>   writeln("arr1.wrap.arr.ptr = ", arr1.wrap.arr.ptr);
>   writeln("arr2.wrap.arr.ptr = ", arr2.wrap.arr.ptr); // identical
>   assert(arr2[0] == 0); // fails
> }
>
> struct IntegerArray
> {
>   int[] arr;
>   alias arr this;
>   this(int l)
>   {
>   arr = new int[l];
>   }
> }
>
> struct WrapIntegerArray
> {
>   auto wrap = IntegerArray(5); // This is CTFE! :-(
>   alias wrap this;
> }

All struct and class members which are directly initialized must have their
values known at compile-time. For structs, that's what goes in the init
value for the type. A side effect of this is that it's usually a bad idea to
directly initialize dynamic arrays which are member variables. You need to
do the initialization in a constructor. And for structs, if you need a
no-arg constructor, then you'll need to use a factory function (since
structs can't have no-arg constructors). e.g.

struct WrapIntegerArray
{
IntegerArray wrap;
alias wrap this;

this(int len)
{
wrap = IntegerArray(len);
}
}

or

struct WrapIntegerArray
{
IntegerArray wrap;
alias wrap this;

static make()
{
WrapIntegerArray retval;
retval.wrap = IntegerArray(5);
return retval;
}
}

So, you could then have something like

auto arr1 = WrapIntegerArray(5);
arr1[0] = 42;

or

auto arr1 = WrapIntegerArray.make();
arr1[0] = 42;

but if you use the init value (which is what you get if you let the type be
default-initialized), then you'll have to first do something to allocate the
dynamic array if you want to be able to index it, since if you don't give it
a value at compile-time, it's null (and you don't want to give it a value at
compile-time, because then every default-initialized struct of that type
will refer to the same dynamic array). Of course, you could always just
append values, and the dynamic array will be allocated and grow accordingly,
but that's obviously not the same as allocating it up front to have a
specific length.

- Jonathan M Davis





Re: A question about postblit constructor

2019-11-06 Thread Jonathan M Davis via Digitalmars-d-learn
On Tuesday, November 5, 2019 5:09:15 AM MST Ferhat Kurtulmuş via 
Digitalmars-d-learn wrote:
> On Tuesday, 5 November 2019 at 12:06:44 UTC, Ferhat Kurtulmuş
>
> wrote:
> > On Tuesday, 5 November 2019 at 11:20:47 UTC, Mike Parker wrote:
> >> On Tuesday, 5 November 2019 at 10:32:03 UTC, Ferhat Kurtulmuş
> >>
> >> wrote:
> [...]
> >>
> >> I meant the example as an answer to your statement, "I wonder
> >> how new memory is allocated without an explicit malloc here".
> >> The postblit was intended as a chance to "fixup" everything
> >> when you needed a deep copy. The new struct is initialized as
> >> a shallow copy, so when you enter into the postblit, the
> >> pointer is already pointing at the original location. Without
> >> assigning it a new malloc'ed address, your memcpy was
> >> essentially overwriting the original location with its own
> >> data.
> >
> > What I need was a deep copy, and I thought I could have done it
> > using postblit constructor. Thanks for clarification.
>
> I think copy constructor is less confusing though.

It's pretty simple really. The compiler takes care of copying everything and
then the postblit constructor only has to worry about stuff where a deep
copy is needed. It's called a postblit, because it's run after the bit
blitting that's used to do a shallow copy. If an object has no postblit
constructor and has no member variables with postblit constructors, then all
a copy does is blit. When a D object with a postblit constructor is run, you
get three steps:

1. A shallow copy of the object is made by bit blitting it.
2. Then the postblit constructors (if any) of the member variables are run.
3. Then the postblit constructor for the object is run.

The result is that in your average postblit constructor, you only have to
deal with a portion of the member variables, whereas for a copy constructor,
you're forced to deal with _every_ member. In principle, postblit
constructors are actually a great idea, because they make it so that you
only have to worry about the members that actually need deep copies.

Where they fall apart is with modifiers like const. Because a postblit
constructor does a shallow copy and then you modify it, it requires
modification, which just doesn't work with const. So, while postblit
constructors were a good idea with D1 (which was a much simpler language),
with D2 (which added const as we know it), they've been far more of a
problem, which is why there was a DIP not long ago to add copy constructors
to D to replace postblit constructors. Because of how long postblit
constructors have been around, they may never actually be deprecated, but
ideally, newer code would use copy constructors, and over time, postblit
constructors wouldn't be used anymore.

Fortunately, D2 has fantastic metaprogramming, so it's actually possible to
write copy constructors that copy all of the members without having to
explicitly copy all of them by name. So, the main benefit of the postblit
constructor (not needing to explicitly copy everything) isn't as much of an
improvement as it originally was.

DIP: https://github.com/dlang/DIPs/blob/master/DIPs/accepted/DIP1018.md

It looks like the release that added copy constructors to the compiler was
2.086 back in May:

https://dlang.org/changelog/2.086.0.html
https://dlang.org/changelog/2.086.0.html#copy_constructor

So, while it's definitely useful for you to understand postblit
constructors, any code you're writing now should probably use copy
constructors. So, if you have a good understanding of copy constructors and
are having trouble with postblit constructors, presumably, that's an
improvement for you, though you may still need to deal with postblit
constructors in existing code that other people have written.

- Jonathan M Davis






Re: No UFCS with nested functions?

2019-11-05 Thread Jonathan M Davis via Digitalmars-d-learn
On Tuesday, November 5, 2019 9:16:27 AM MST ixid via Digitalmars-d-learn 
wrote:
> On Monday, 4 November 2019 at 20:46:41 UTC, H. S. Teoh wrote:
> > On Mon, Nov 04, 2019 at 07:51:26PM +, Tobias Pankrath via
> >
> > Digitalmars-d-learn wrote:
> >> Why does the following not work? It works, if I move the
> >> 'prop' out of 'foo'.
> >
> > UFCS is only supported for module-level functions, as far as I
> > know.
> >
> >> ---
> >> struct S {
> >>
> >>ubyte[12] bar;
> >>
> >> }
> >>
> >> bool foo (ref S s)
> >> {
> >>
> >>static bool prop(const(ubyte)[] f) {
> >>
> >>   return f.length > 1;
> >>
> >>}
> >>
> >>return s.bar[].prop;
> >>
> >> }
> >> ---
> >
> > [...]
> >
> >
> > T
>
> Is this a necessary limitation? It feels inconsistent and clunky.

It's explained at the end of this section of the documentation:

https://dlang.org/spec/function.html#pseudo-member

- Jonathan M Davis





Re: Is there any writeln like functions without GC?

2019-11-02 Thread Jonathan M Davis via Digitalmars-d-learn
On Thursday, October 31, 2019 9:11:42 AM MDT Ferhat Kurtulmuş via 
Digitalmars-d-learn wrote:
> On Thursday, 31 October 2019 at 13:46:07 UTC, Adam D. Ruppe wrote:
> > On Thursday, 31 October 2019 at 03:56:56 UTC, lili wrote:
> >> Hi:
> >>why writeln need GC?
> >
> > It almost never does, it just keeps the option open in case
> >
> > * it needs to throw an exception (like if stdout is closed)
> >
> > * you pass it a custom type with toString that uses GC
> >
> > @nogc is just super strict and doesn't even allow for rare
> > cases.
>
> It would be nice if one reimplement writeln of Phobos by
> bypassing gc and use a custom nogc exception as described here*?
> Of course I can imagine that it would be a breaking change in the
> language and requires so much work for it to be compatible with
> other std modules/language features.
>
> *:
> https://www.auburnsounds.com/blog/2016-11-10_Running-D-without-its-runtime
> .html

You can always just use printf.

- Jonathan M Davis






Re: Documentation: is it intentional that template constraints are displayed after the signature?

2019-11-01 Thread Jonathan M Davis via Digitalmars-d-learn
On Friday, November 1, 2019 11:23:42 AM MDT Ali Çehreli via Digitalmars-d-
learn wrote:
> On 11/01/2019 09:33 AM, Paul Backus wrote:
>  > On Friday, 1 November 2019 at 15:29:24 UTC, Ali Çehreli wrote:
>  >> Apparently, it's the version for static arrays. However, I don't think
>  >> the template constraint is doing anything there because if T matches a
>  >> static array (of the form U[n]), then T is not a struct anyway.
>  >>
>  >> Ali
>  >
>  > `T : U[n]` could also be matched by a struct with an `alias this` to a
>  > static array member.
>  >
>  > Example: https://run.dlang.io/is/NgRU94
>
> Thanks. Is it special to destroy() to care for that case or should all
> our algorithms be on the watchout for such structs? I'm sure I would be
> missing that specialization if I ever needed to write similar
> specializations for an algorithm.

Pretty much any time that a template constraint uses an implicit conversion,
the code needs to be careful. Typically, the code then needs to force the
conversion and then operate on the exact type in order to avoid subtle bugs.
Honestly, in general, I'd argue that it's better to just not accept implicit
conversions with templates.

In this particular case, what it should probably be doing is just use
__traits(isStaticArray, ...) in the template constraint, but whoever wrote
that overload took a different tact (though depending on how old that trait
is and how old that overload of destroy is, the trait may or may not have
been an option when that overload of destroy was written).

- Jonathan M Davis






Re: Using a char value >= 128

2019-10-27 Thread Jonathan M Davis via Digitalmars-d-learn
On Sunday, October 27, 2019 6:44:05 AM MDT Per Nordlöw via Digitalmars-d-
learn wrote:
> In which circumstances can a `char` be initialized a non-7-bit
> value (>= 128)? Is it possible only in non-@safe code?
>
> And, if so, what will be the result of casting such a value to
> `dchar`? Will that result in an exception or will it interpret
> the `char` using a 8-bit character encoding?
>
> I'm asking because I'm pondering about how to specialize the
> non-7-bit `needle`-case of the following array-overload of
> `startsWith` when `T` is `char`:
>
> bool startsWith(T)(scope const(T)[] haystack,
> scope const T needle)
> {
>  static if (is(T : char)) { assert(needle < 128); } // TODO
> convert needle to `char[]` and call itself
>  if (haystack.length >= 1)
>  {
>  return haystack[0] == needle;
>  }
>  return false;
> }

char is a value above 127 all the time, because specific values above 127
are used as the first byte in a multibyte code point in UTF-8. Also, as Adam
points out, the default value for char is 255 (in order to specifically give
it an invalid value).

That being said, it doesn't make sense to use startsWith with a single char
which isn't ASCII, because no such char would be valid UTF-8 on its own.

- Jonathan M Davis






Re: About the in expression, Why can't use with array.

2019-10-24 Thread Jonathan M Davis via Digitalmars-d-learn
On Thursday, October 24, 2019 8:40:59 PM MDT lili via Digitalmars-d-learn 
wrote:
> On Thursday, 24 October 2019 at 22:40:31 UTC, Jonathan M Davis
>
> wrote:
> > On Thursday, October 24, 2019 7:04:56 AM MDT Paul Backus via
> >
> > Digitalmars-d- learn wrote:
> >> On Thursday, 24 October 2019 at 12:58:11 UTC, lili wrote:
> >> > Hi:
> >> >In Dlang where is strange design. The in expression can
> >> >
> >> > only
> >> >
> >> > use to associative array, why array can not use in
> >> > expression.
> >>
> >> Checking for the presence of an item in an array requires a
> >> linear search. You can do it with
> >> std.algorithm.searching.canFind:
> >>
> >> https://dlang.org/phobos/std_algorithm_searching.html#.canFind
> >
> > In particular, the reason that linear search is considered
> > unacceptable for
> > in is so that generic code can rely on its performance. The
> > idea is that
> > types shouldn't implement the in operator unless they can do so
> > with
> > O(log n) or better (O(log n) being what it costs to get to an
> > item in a
> > balanced binary tree like a red-black tree). That way, when you
> > calculate
> > the complexity of any algorithm using in, you can assume that
> > it's O(log n)
> > at worst. Having it be O(n) instead (like it would be for an
> > array) would
> > drastically increase the complexity of the algorithm and make
> > it take much
> > longer when processing a large number of items. And the
> > standard library
> > provides functions like canFind or find for finding elements in
> > an array, so
> > having in work with arrays wouldn't add any functionality. It
> > would
> > basically just change the syntax you'd use for finding an
> > element in an
> > array.
> >
> > - Jonathan M Davis
>
>   This reason somewhat farfetched, Think about that if this reason
> is right, the operator overload can not use。 because same
> operator on different type expression same mean but different
> implementation and complexity。 so why operator overload can don't
> care about implementation but 'in' operator need。

std.container specifically discusses the expected, worse-case complexity of
(i.e. Big-O complexity) all of the various container-related functions -
including the in operator:

https://dlang.org/phobos/std_container.html

As such, anyone writing an algorithm using any of those functions can rely
on the worst-case complexity of each operation and accurately calculate the
overall, worst-case complexity of their algorithm. Sure, someone could
choose to overload the in operator in a manner which does not follow those
rules, but it's considered bad practice to do so, and no official
implementation of any kind is going to provide a function with a worse
complexity than what std.container lays out - and that includes operators
for the built-in types like a dynamic array. This is basically the same
approach that C++ takes with its STL containers and their related
operations.

- Jonathan M Davis






Re: About the in expression, Why can't use with array.

2019-10-24 Thread Jonathan M Davis via Digitalmars-d-learn
On Thursday, October 24, 2019 7:04:56 AM MDT Paul Backus via Digitalmars-d-
learn wrote:
> On Thursday, 24 October 2019 at 12:58:11 UTC, lili wrote:
> > Hi:
> >In Dlang where is strange design. The in expression can only
> >
> > use to associative array, why array can not use in expression.
>
> Checking for the presence of an item in an array requires a
> linear search. You can do it with std.algorithm.searching.canFind:
>
> https://dlang.org/phobos/std_algorithm_searching.html#.canFind

In particular, the reason that linear search is considered unacceptable for
in is so that generic code can rely on its performance. The idea is that
types shouldn't implement the in operator unless they can do so with
O(log n) or better (O(log n) being what it costs to get to an item in a
balanced binary tree like a red-black tree). That way, when you calculate
the complexity of any algorithm using in, you can assume that it's O(log n)
at worst. Having it be O(n) instead (like it would be for an array) would
drastically increase the complexity of the algorithm and make it take much
longer when processing a large number of items. And the standard library
provides functions like canFind or find for finding elements in an array, so
having in work with arrays wouldn't add any functionality. It would
basically just change the syntax you'd use for finding an element in an
array.

- Jonathan M Davis





Re: Why isn't skipOver(string, string) nothrow?

2019-10-22 Thread Jonathan M Davis via Digitalmars-d-learn
On Tuesday, October 22, 2019 4:27:59 PM MDT Per Nordlöw via Digitalmars-d-
learn wrote:
> On Tuesday, 22 October 2019 at 15:39:17 UTC, Adam D. Ruppe wrote:
> > On Tuesday, 22 October 2019 at 15:33:05 UTC, Per Nordlöw wrote:
> >> Why isn't a call to
> >>
> >> skipOver(string, string)
> >>
> >> nothrow?
> >
> > without really looking, probably because of invalid utf
> > sequences potentially throwing. Using the .representation
> > thingy might help if im right about this.
> >
> > A good way to try this is to edit your copy of the Phobos file
> > and add nothrow to it. That should give an error inside that is
> > more descriptive. (I would love if the compiler would do this
> > automatically, we were talking about maybe making that mod on
> > irc yesterday).
>
> But startsWith(string, string) is nothrow so skipOver should be
> that too.

That's only true with startsWith, because it avoids decoding in the case
where the two strings have the same encoding (e.g. it's not nothrow if you
compare a dstring and a string). Presumably, skipOver could be made to do
the same, but no one has done so.

- Jonathan M Davis






Re: Why isn't skipOver(string, string) nothrow?

2019-10-22 Thread Jonathan M Davis via Digitalmars-d-learn
On Tuesday, October 22, 2019 9:33:05 AM MDT Per Nordlöw via Digitalmars-d-
learn wrote:
> Why isn't a call to
>
>  skipOver(string, string)
>
> nothrow?
>
> I see no reason why it shouldn't be.
>
> Further, this test should be qualifyable as nothrow:
>
> @safe pure /* TODO nothrow @nogc */ unittest
> {
>  import std.algorithm.searching : skipOver;
>  auto x = "beta version";
>  assert(x.skipOver("beta"));
>  assert(x == " version");
> }

Almost anything involving strings isn't going to be nothrow, because front
and popFront throw on invalid UTF. To really fix that, we'd need to get rid
of auto-decoding. You can use std.utf.byDchar to wrap the string in a range
of dchar which replaces invalid Unicode with the replacement character
instead, which means that no exception gets thrown, but it also means that
if you hadn't previously validated the Unicode, you could end up processing
invalid Unicode without realizing it. How much that matters depends on what
you're doing. Ideally, all strings would just be validated when they were
created, and then it wouldn't be an issue, but any code that decodes the
code points would still have to deal with invalid Unicode in some manner
(though if we decided that it was the responsibility of the caller to always
validate the Unicode first, then we could use assertions). For better or
worse, the chosen solution when ranges were first put together was to throw
on invalid Unicode, which basically makes it impossible for functions that
process strings to be nothrow unless they go to the extra effort working
around auto-decoding. If we're ever able to remove auto-decoding, then
that's no longer an issue for all string processing functions, but it's
still going to be an issue for any code that calls functions like decode or
stride. They're either going to throw or replace invalid Unicode with the
replacement character. Which approach is better depends on the code.

In any case, as long as auto-decoding is a thing, you'll have to use
wrappers like byDchar or byCodeUnit if you want much of anything involving
strings to be nothrow.

- Jonathan M Davis






Re: Some questions about GC

2019-10-18 Thread Jonathan M Davis via Digitalmars-d-learn
On Friday, October 18, 2019 10:54:55 AM MDT Roland Hadinger via Digitalmars-
d-learn wrote:
> These questions probably need some context: I'm working on an
> interpreter that will manage memory via reference counted struct
> types. To deal with the problem of strong reference cycles
> retaining memory indefinitely, weak references or recursive
> teardowns have to be used where appropriate.
>
> To help detect memory leaks from within the interpreter, I'd also
> like to employ core.memory.GC in the following fashion:
>
> * keep core.memory.GC off (disabled) by default, but nonetheless
> allocate objects from GC memory
> * provide a function that can find (and reclaim) retained
> unreachable object graphs that contain strong reference cycles
> * the main purpose of this function is to find and report such
> instances, not to reclaim memory. Retained graphs should be
> reported as warnings on stderr, so that the program can be fixed
> manually, e.g. by weakening some refs in the proper places
> * the function will rely on GC.collect to find unreachable objects
> * the function will *always* be called implicitly when a program
> terminates
> * the function should also be explicitly callable from any point
> within a program.
>
> Now my questions:
>
> Is it safe to assume that a call to GC.collect will be handled
> synchronously (and won't return early)?

D's GC is a stop-the-world GC. Every thread managed by the GC is stopped
when a thread runs a collection.

> Is there a way to ensure that GC.collect will never run unless
> when called explicitly (even in out of memory situations)?

The GC only runs a collection either when you explicitly tell it to or when
you try to allocate memory using the GC, and it determines that it should
run a collection. Disabling the GC normally prevents a collection from
running, though per the documentation, it sounds like it may still run if
the GC actually runs out of memory. I had thought that it prevented
collections completely, but that's not what the documentation says. I don't
know what the current implementation does.

> Is it possible and is it OK to print to stderr while the GC is
> collecting (e.g. from @nogc code, using functions from
> core.stdc.stdio)?

No code in any thread managed by the GC is run while a collection is running
unless it's code that's triggered by the collection itself (e.g. a finalizer
being called on an object that's being collected - and even that isn't
supposed to access GC-allocated objects, because the GC might have already
destroyed them - e.g. in the case of cycle). If you want code to run at the
same time as a GC collection, it's going to have to be in a thread that is
not attached to the GC, and at that point, you shouldn't be accessing
_anything_ that's managed by the GC unless you have a guarantee that what
you're accessing won't be collected. And even then, you shouldn't be
mutating any of it.

Also, @nogc doesn't say anything about whether the code accesses
GC-allocated objects. It just means that it's not allowed to access most GC
functions, which usually just means that it doesn't allocate anything using
the GC and that it doesn't risk running a collection. So, just because a
function is @nogc doesn't necessarily mean that it's safe to run it from a
thread that isn't managed by the GC while a collection is running.

> Could I implement my function by introducing a shared global flag
> which is set prior to calling GC.collect and reset afterwards, so
> that any destructor can determine whether has been invoked by a
> "flagged" call to GC.collect and act accordingly?

You should be able to do that, but then the destructor can't be pure (though
as I understand it, there's currently a compiler bug with pure destructors
anyway which causes them to not be called), and when a destructor is run as
a finalizer, it shouldn't be accessing any other GC-allocated objects,
because the GC might have actually destroyed them already at that point.
Finalizers really aren't supposed to doh much of anything other than
managing what lives in an object directly or managing non-GC-allocated
resources. Regardless, anything that really should be operating as a
destructor rather than a finalizer has to live on the stack, since
finalizers won't be run until a collection occurs. If you're explicitly
running them yourself via your own reference counting, then you don't have
that problem, but if there's any chance that a destructor is going to be run
as a finalizer by the GC, then you have to write your destructors /
finalizers with the idea that that could happen.

> Alternatively: do I need to implement such a flag, or is there
> already a way in which a destructor can determine whether it has
> been invoked by the GC?
>
> Thanks for any help!

Honestly, the way things are set up, destructors aren't supposed to know or
care about whether they're being run by the GC as a finalizer. So, the GC
isn't going to provide that kind of functionality. What you're 

Re: How Different Are Templates from Generics

2019-10-12 Thread Jonathan M Davis via Digitalmars-d-learn
On Saturday, October 12, 2019 9:48:02 PM MDT jmh530 via Digitalmars-d-learn 
wrote:
> On Saturday, 12 October 2019 at 21:44:57 UTC, Jonathan M Davis
>
> wrote:
> > [snip]
>
> Thanks for the reply.
>
> As with most people, I don't write a lot of D code that uses
> classes that much.
>
> The use case I'm thinking of is with allocators, which - to be
> honest - is not something I deal with much in my own code.
> Basically, some of the examples have stuff like
> ScopedAllocator!Mallocator, which would imply that there is a
> different ScopedAllocator for each allocator. However, if you
> apply Java's generics, then you would just have one. Not sure if
> it would make any kind of difference in real-life code, but still
> interesting to think about.

I wouldn't think that there would be enough different allocator types to
matter much. Certainly, the amount of code that gets generated by templates
for dealing with stuff like ranges would dwarf it. If program size really
becomes a problem, then examining how code uses templates and trying to
reduce how much they're used could certainly have an impact, but I'd expect
it to be fairly rare that attempting to emulate Java's generics would help
much - especially since it would only work when classes were involved. The
main place that such an approach would have much chance of having an impact
would be with regards to container implementations when the code puts a lot
of different types of class objects inside of containers, and even that
would easily be dwarfed by all of the other template usage in your typical D
program. For Java's approach to make much sense, you'd probably have to be
writing very Java-like code.

- Jonathan M Davis





Re: How Different Are Templates from Generics

2019-10-12 Thread Jonathan M Davis via Digitalmars-d-learn
On Saturday, October 12, 2019 2:11:28 PM MDT jmh530 via Digitalmars-d-learn 
wrote:
> On Friday, 11 October 2019 at 17:50:42 UTC, Jonathan M Davis
>
> wrote:
> > [snip]
>
> A very thorough explanation!
>
> One follow-up question: would it be possible to mimic the
> behavior of Java generics in D?

Yes, but it's unlikely that it would make any sense to do so. You'd
basically have to do something like

auto foo(T)(T obj)
if(is(T : Object) && !is(T == Object))
{
return foo(cast(Object)obj);
}

auto foo(Object obj)
{
...
}

And for containers, you'd basically end up with a templated container that
was just a wrapper around a non-templated container that operated on Object.

If you went with such an approach, you'd get less code in the binary, but
you'd also end up with a deeper call stack because of all of the wrappers
needed to add the casts for you.

However, since Object can't do much of anything, having code that operates
on Object isn't usually very useful. You could have the code use a different
base class that had whatever operations you wanted, but you're still adding
a fair bit of extra machinery just to avoid a few template instantiations.

And since idiomatic D doesn't use classes much (rather, best practice is to
use a struct unless you need polymorphism or you need something to always be
a reference type), and it uses templates quite heavily (that's especially
true with range-based code), it would be pretty bizarre to try and use
Java's approach in D.

- Jonathan M Davis





Re: selective tests

2019-10-12 Thread Jonathan M Davis via Digitalmars-d-learn
On Saturday, October 12, 2019 2:18:02 AM MDT Martin Brezeln via Digitalmars-
d-learn wrote:
> Is it possible to execute only certain modules or tests which are
> defined in certain directories?
>
> For example, in go one can run "go test ./XYZ" to execute ony
> tests in ./XYZ or "go test ./..." to execute all the tests in and
> under the current directory.
>
> Please don't get me wrong, i do not wish to start a discussion
> about doing things the "go way". I am asking if there is a way to
> achieve a similar result with dub (or maybe without dub?
> 樂)

The default test runner does not support running only some of the tests. It
simply runs all of the unittest blocks in the binary prior to running main,
and tests only get skipped when they're either not compiled in or when a
previous unittest block in that module failed.

You could set up your build so that you had targets which only compiled
specific directories so that the only unit tests that were run were the ones
in those directories, but I don't think that it's possible to do anything
like that with dub. Certainly, if it is, it would be a royal pain to set up.

Really, if you want to control which tests get run instead of simply always
running them all, then you'll need to use an alternate test runner which
supports that. There are a few test runners available on code.dlang.org, and
I expect that at least one of them supports that (probably multiple do).

- Jonathan M Davis






Re: _getmaxstdio / _setmaxstdio

2019-10-11 Thread Jonathan M Davis via Digitalmars-d-learn
On Thursday, October 10, 2019 5:03:29 PM MDT Damian via Digitalmars-d-learn 
wrote:
> Missing _getmaxstdio / _setmaxstdio?
> I'd like to try and increase the limit of open files without
> resorting to Windows API, is it possible or will I have to resort
> to the WinAPI to achieve this?

Phobos doesn't have anything like that (and if it did, it would just be a
cross-platform wrapper which used the Windows API on Windows whatever the
equivalent would be on other platforms - assuming that there even is an
equivalent on other platforms). So, unless there's a library on
code.dlang.org which has such a wrapper, you'll need to use the Windows API
directly. But it's not like that would be hard. All you'd need to do would
be to declare the appropriate function declaration (which would probably be
extern(Windows) in this case) and make sure that you're linked against the
appropriate library. Given that the header is stdio.h, that would presumably
be Microsoft's C runtime library, which probably means that you'd need to
tell dmd to use Microsoft's C runtime and not dmc's C runtime.

- Jonathan M Davis





Re: How Different Are Templates from Generics

2019-10-11 Thread Jonathan M Davis via Digitalmars-d-learn
On Friday, October 11, 2019 12:09:20 PM MDT Just Dave via Digitalmars-d-
learn wrote:
> Thanks for the thorough explanation. Most of that is how I was
> thinking it worked. However, that leaves me perplexed. If
> templates just generate code then how come:
>
> Wouldnt..
>
>  class SomeClass(T) : ISomeInterface!T
>
> and..
>
>  class SomeOtherClass(T) : ISomeInterface!T
>
> ...generate two different interfaces? Two interfaces that do the
> same thing, but two interfaces nonetheless? I assume each type in
> D has some form of type id underlying everything, which wouldn't
> that make the follow:
>
>  if (instance1 is ISomeInterface)
>  {
>  Console.WriteLine("Instance1 is interface!");
>  }
>
> fail? Or is there some extra magic that is making it work with my
> experiments?

You get a different template instantiation for each set of template
arguments. So, if you have ISomeInterface!int, and you use
ISomeinterface!int somewhere else, because they're both instantiating
ISomeInterface with the same set of template arguments, you only get one
instantiation. So,

class SomeClass : ISomeInterface!int

and

class SomeOtherClass : ISomeInterface!int

would both be implementing the exact same interface. And if you then have

class SomeClass(T) : ISomeInterface!T

and

class SomeOtherClass(T) : ISomeInterface!T

then SomeClass!int and SomeOtherClass!int would both be implementing the
same interface, because in both cases, it would be ISomeInterface!int.
SomeClass!int and SomeOtherClass!float would not be implementing the same
interface, because it would be ISomeInterface!int and ISomeInterface!float,
but ISomeInterface!int doesn't result in multiple instantiations even if
it's used in different parts of the code.

- Jonathan M Davis





Re: How Different Are Templates from Generics

2019-10-11 Thread Jonathan M Davis via Digitalmars-d-learn
On Friday, October 11, 2019 8:43:49 AM MDT Just Dave via Digitalmars-d-learn 
wrote:
> I come from both a C++ and C# background. Those have been the
> primary languages I have used. In C# you can do something like
> this:
>
>  public interface ISomeInterface
>  {
>   T Value { get; }
>  }
>
>  public class SomeClass : ISomeInterface
>  {
>   T Value { get; set; }
>  }
>
>  public class SomeOtherClass : ISomeInterface
>  {
>  T Value { get; set; }
>  }
>
>  public static class Example
>  {
>  public static void Foo()
>  {
>  var instance1 = new SomeClass(){ Value = 4; };
>  var instance2 = new SomeClass(){ Value = 2; };
>
>  if (instance1 is ISomeInterface)
>  {
>  Console.WriteLine("Instance1 is interface!");
>  }
>
>  if (instance2 is ISomeInterface)
>  {
>  Console.WriteLine("Instance2 is interface!");
>  }
>  }
>  }
>
> Expected output is both WriteLines get hit:
>
>  Instance1 is interface!
>
>  Instance2 is interface!
>
>
> So now the 'D' version:
>
>  interface ISomeInterface(T)
>  {
>   T getValue();
>  }
>
>  class SomeClass(T) : ISomeInterface!T
>  {
>  private:
>   T t;
>
>  public:
>   this(T t)
>   {
>  this.t = t;
>   }
>
>   T getValue()
>   {
> return t;
>   }
>  }
>
>  class SomeOtherClass(T) : ISomeInterface!T
>  {
>  private:
>   T t;
>
>  public:
>   this(T t)
>   {
>  this.t = t;
>   }
>
>   T getValue()
>   {
>   return t;
>   }
>  }
>
> ...which seems to work the same way with preliminary testing. I
> guess my question is...templates are different than generics, but
> can I feel confident continuing forward with such a design in D
> and expect this more or less to behave as I would expect in C#?
> Or are there lots of caveats I should be aware of?

Generics and templates are syntactically similiar but are really doing very
different things.

Generic functions and types operate on Object underneath the hood. If you
have Container and Container, you really just have
Container with some syntactic niceties to avoid explicit casts. You
get type checks to ensure that Container isn't given a Bar unless Bar
is derived from Foo, and the casts to and from Object when giving
Container a Foo are taken care of for you, but it's still always
Container underneath the hood.

In the case of Java, the type of T in Container or foo() is truly only
a compile time thing, so the bytecode only has Container and no clue
what type is actually supposed to be used (the casts are there where the
container or function is used, but the container or function has no clue
what the type is; it just sees Object). That makes it possible to cheat with
reflection and put something not derived from Foo in Container but
will then usually result in runtime failures when the casts the compiler
inserted are run. C# doesn't have that kind of type erasure in that the
information that Container contains Foo rather than Object is
maintained at runtime, but you still have a Container. It's just a
Container with some metadata which keeps track of the fact that for
this particular object of Container, Object is always supposed to be
a Foo. As I'm a lot less familiar with C# than Java, I'm not all that
familiar with what the practical benefits that gives are, though I'd expect
that it would mean that reflection code would catch when you're trying to
put a Bar into Container and wouldn't let you.

Note that for generics to work, they have to a common base type, and you
only ever get one version of a generic class or function even if it gets
used with many different types derived from Object. For a primitive type
like int or float (as well as for structs in the case of C#), they have to
be put into a type derived from Object in order to be used with generics (as
I expect you're aware, C# calls this boxing and unboxing). Templates don't
act like this at all.

Templates are literally templates for generating code. A template is nothing
by itself. Something like

struct Container(T)
{
T[] data;
}

or

T foo(T)(T t)
{
return t;
}

doesn't result in any code being in the binary until unless template is
instantiated with a specific type, and when that template is instantiated,
code is generated based on the type that it's instantiated with. So,
Container!int and Container!Foo result in two different versions of
Container being generated and put in the binary - one which operates on int,
and one which operates on Foo. There is no conversion to Object going on
here. The code literally uses int and Foo directly and is generated
specifically for those types. Not only does that mean that the generated
code can be optimized for the specific type rather than being for any
Object, but it also means that the code itself could do 

Re: C#'s 'is' equivalent in D

2019-10-10 Thread Jonathan M Davis via Digitalmars-d-learn
On Thursday, October 10, 2019 9:47:58 AM MDT Just Dave via Digitalmars-d-
learn wrote:
> In C# you can do something like:
>
>
>  if (obj is Person)
>  {
>  var person = obj as Person;
>  // do stuff with person...
>  }
>
> where you can check the type of an object prior to casting. Does
> D have a similar mechanism? It's so widely useful in the C# realm
> that they even added syntactic sugar to allow:
>
>  if (obj is Person person)
>  {
>  // do stuff with person...
>  }
>
> I would presume since D has reference objects there must exist
> some mechanism for this...

D's solution is basically the same as C++'s solution. You cast and then
check whether the result is null. So,

if(cast(Person)obj !is null)
{
}

or since using a pointer or reference in an if condition checks whether it's
null or not

if(cast(Person)obj)
{
}

and you can even declare a variable that way if you want. e.g.

if(auto person = cast(Person)obj)
{
}

When D's is is used to compare two objects, it checks whether they're equal
bitwise. So, it's typically used for comparing pointers or references for
equality (whereas using == with references would compare the objects
themselves for equality).

- Jonathan M Davis





Re: Dynamic Arrays as Stack and/or Queue

2019-10-08 Thread Jonathan M Davis via Digitalmars-d-learn
On Tuesday, October 8, 2019 2:42:22 PM MDT mipri via Digitalmars-d-learn 
wrote:
> On Tuesday, 8 October 2019 at 10:48:45 UTC, Jonathan M Davis
>
> wrote:
> > The result of this is that code like
> >
> > stack.popBack();
> > stack ~= foo;
> > stack ~= bar;
> > stack.popBack();
> > stack ~= baz;
> >
> > will end up allocating all over the place. Every time you
> > append to the array after shrinking it, you're going to end up
> > with the GC allocating a new block of memory instead of
> > appending in place.
>
> Thanks as well! I thought the code I posted would only allocate
> when
> the array ran out of capacity. And I see how, even if that's
> worked
> around with assumeSafeAppend, it becomes a bad idea to define the
> stack functions against `ref T[]` as this makes it too easy for
> other
> code to slice the array and cause the bugs that assumeSafeAppend
> allows.

Technically, it only does allocate when it runs out of capacity. However, if
you do something like

arr = arr[0 .. $ - 1];

then arr.capacity will either be the length of the array or 0 (I don't
remember which off the top of my head), because the capacity is calculated
based on how much space is available after the end of the dynamic array. How
much space is available at the end of the memory block is irrelevant unless
the dynamic array includes the last element that any dynamic array has ever
had within that memory block. If it's at the end, and the capacity is
greater than the length of the array, then the array can expand in place (up
to the difference between the length and the capacity). But if there is no
extra room on the end, or if the current dynamic array is not at the end,
then the capacity will reflect that. The same goes if the dynamic array is
actually a slice of memory that wasn't allocated by the GC for dynamic
arrays. IIRC, a dynamic array which is a slice of malloc-ed memory will
always give a capacity of 0, but regardless of whether it's actually 0, it's
never more than the length of the dynamic array, because there is no extra
capacity to grow into, since the memory was not allocated by the GC for use
by a dynamic array.

All of the various dynamic array functions work the same regardless of what
kind of memory is backing the dynamic array. It's just that if the memory
wasn't allocated by the GC for dynamic arrays, then when you call the
dynamic array function or property, then the GC treats it the same as it
would treat a dynamic array that referred to the end of the block of memory
(and thus wouldn't have any extra capacity).

You should always be able to call capacity on a dynamic array and see
whether appending to would then result in a reallocation or not.

Either way, because assumeSafeAppend resets the metadata so that the dynamic
array it's given is considered to be the farthest dynamic array within the
block of memory, after calling assumeSafeAppend, capacity will reflect that.

- Jonathan M Davis





Re: Ranges to deal with corner cases and "random access"

2019-10-08 Thread Jonathan M Davis via Digitalmars-d-learn
On Tuesday, October 8, 2019 9:40:33 AM MDT Paul Backus via Digitalmars-d-
learn wrote:
> On Sunday, 6 October 2019 at 20:34:55 UTC, Brett wrote:
> > If it can be done and make to work well with ranges it would
> > allow many algorithms to be very easily expressed and make
> > ranges more powerful.
>
> You can make it a range by adding an "alias this" to the original
> array:
>
> struct ExtendedArray(T)
> {
>  T[] a;
>  alias a this;
>
>  T opIndex(int i)
>  {
>  if (i < 0) return a[0];
>  else if (i >= a.length) return a[$-1];
>  else return a[i];
>  }
> }
>
> Full example: https://run.dlang.io/is/2x6LKD

It would be less error-prone to just implement the appropriate functions on
the struct. alias this and generic code are a terrible combination. It's way
too easy for a type to pass a template constraint thanks to alias this and
then have trouble because it passed based on the implicit conversion, but
the conversion wasn't forced in the code using the type. You can get some
really subtle problems if the code converts to the alias in some cases but
not in others.

- Jonathan M Davis





Re: Dynamic Arrays as Stack and/or Queue

2019-10-08 Thread Jonathan M Davis via Digitalmars-d-learn
On Monday, October 7, 2019 1:16:31 PM MDT IGotD- via Digitalmars-d-learn 
wrote:
> On Monday, 7 October 2019 at 17:36:09 UTC, Ferhat Kurtulmuş wrote:
> >> I'm not talking about memory deletion. I'm talking about push,
> >> pop, enqueue, and dequeue behavior. I'd assume in a garbage
> >> collected language letting the reference float off should be
> >> picked up by the GC.
> >
> > I'm sorry. Writing on my mobile phone. Maybe this is what you
> > are looking for
> > https://dlang.org/phobos/std_container_dlist.html
>
> I think what he is looking for are the general pop_front,
> push_front, pop_back and push_back that you would find in
> virtually any C++ STL container algorithms like list, vector or
> map.
>
> I think this is a good question as I don't really see any good
> example in the documentation of the dynamic arrays about this.
> This is very common use case for arrays. Is there any D
> equivalent?

Pushing and popping like you'd do with a stack or queue can certainly be
done with a dynamic array, but D's dynamic arrays don't work very well that
way without wrapping them. Shrinking them is easy enough. You just slice the
dynamic array to get another dynamic array which is a slice referring just
to the elements that were sliced. e.g.

auto arr2 = arr1[5 .. $];
assert(arr2.length == arr1.length - 5);
assert(arr2 == arr1[5 .. $]);

If you just want to pop off the first element, you can even use popFront
from std.range.primitives. e.g.

auto arr2 = arr1;
arr2.popFront();
assert(arr2 == arr1[1 .. $]);

Similarly, you could use popBack to pop off the last element. e.g.

auto arr2 = arr1;
arr2.popBack();
assert(arr2 == arr1[0 .. $ - 1]);

~= can be used to append to a dynamic array, but then we start getting into
some issues.

When a dynamic array is appended to, the GC determines whether it has the
capacity to put that element one past the end of that dynamic array within
the block of memory that that dynamic array is a slice of. If the dynamic
array is not a slice of GC-allocated memory for dynamic arrays, then the GC
determines that it can't append in place, so it allocates a new block of
memory, copies the elements to that new block of memory (including the
appendend elements), and then mutates that dynamic array to point to the new
block of memory. Similarly, if the dynamic array refers to a GC-allocated
block of memory with no extra space on the end, the GC will allocate a new
block of memory, copy the elements over, and mutate the dynamic array to
point to the new block of memory. On the other hand, if the memory block was
allocated by the GC for dynamic arrays, and it does have space beyond the
end of that dynamic array, then it will just append in place and adjust the
length member of the dynamic array accordingly. However, the real kicker for
this particular use case is what happens when the dynamic array's last
element is not the last element used within the memory block. e.g.

arr2 = arr1[0 .. $ - 1];
arr2 ~= 42;

or even

arr1 = arr1[0 .. $ - 1];
arr1 ~= 42;

Both of those examples will result in a new block of memory being allocated
when the dynamic array is appended to. That's because the GC is written to
avoid appending into another dynamic array which refers to the same block of
memory. In order for a dynamic array to have capacity to expand into, no
other dynamic array can ever have had any elements beyond the end of that
dynamic array (the metadata keeps track of the farthest that any array has
expanded into the block of memory). Even if there are currently no other
dynamic arrays which refer to that element, it doesn't matter. All the GC
cares about is whether any dynamic array has ever expanded that far into the
memory block.

The result of this is that code like

stack.popBack();
stack ~= foo;
stack ~= bar;
stack.popBack();
stack ~= baz;

will end up allocating all over the place. Every time you append to the
array after shrinking it, you're going to end up with the GC allocating a
new block of memory instead of appending in place.

The solution to this problem is to use the function assumeSafeAppend
(it's in object.d, so it's always available). e.g.

stack.popBack();
stack.assumeSafeAppend();
stack ~= foo;
stack ~= bar;
stack.popBack();
stack.assumeSafeAppend();
stack ~= baz;

This tells the GC to reset the metadata for the block of memory that that
dynamic array is a slice of so that however far the last element of that
dynamic array is is considered to be the last element curently used in the
memory block. That way, when you append to it, it won't think that there are
any other dynamic arrays using that memory, and it will expand the array in
place.

The problem with this (and the reason that assumeSafeAppend is @system) is
that if you ever use it when there are still other dynamic arrays in use
which are slices of that same block of memory and which refer to elements
beyond the end of the dynamic array that you call assumeSafeAppend on,
you're going to get some subtle and nasty 

Re: How does D distnguish managed pointers from raw pointers?

2019-10-04 Thread Jonathan M Davis via Digitalmars-d-learn
On Friday, October 4, 2019 1:22:26 PM MDT Dennis via Digitalmars-d-learn 
wrote:
> On Friday, 4 October 2019 at 19:08:04 UTC, Adam D. Ruppe wrote:
> > (personally though I like to explicitly slice it all the time
> > though, it is more clear and the habit is nice)
>
> Turns out I have this habit as well. I'm looking through some of
> my code and see redundant slicing everywhere.

Really, it should be required by the language, because it's not something
that you want to be hidden. It's an easy source of bugs - especially once
you start passing that dynamic array around. It's incredibly useful to be
able to do it, but you need to be careful with such code. It's the array
equivalent of taking the address of a local variable and passing a pointer
to it around.

IIRC, -dip1000 improves the situation by making it so that the type of a
slice of a static array is scope, but it's still easy to miss, since it only
affects @safe code. It should certainly be possible to slice a static array
in @system code without having to deal with scope, but the fact that
explicit slicing isn't required in such a case makes it more error-prone
than it would be if explicit slicing were required.

- Jonathan M Davis





Re: Using enforce or assert to check for fullness when appending to fixed length array container

2019-10-04 Thread Jonathan M Davis via Digitalmars-d-learn
On Friday, October 4, 2019 4:00:08 AM MDT Per Nordlöw via Digitalmars-d-
learn wrote:
> I have a wrapper container FixedArray at
>
> https://github.com/nordlow/phobos-next/blob/25f4a4ee7347427cebd5cd375c9990
> b44108d2ef/src/fixed_array.d
>
> on top of a static array store that provides the member
>
>  void insertBack(Es...)(Es es) @trusted
>  if (Es.length <= capacity) // TODO use `isAssignable`
>  {
>  import std.exception : enforce;
>  enforce(_length + Es.length <= capacity, `Arguments don't
> fit in array`); // TODO use assert insteead?
>
>  foreach (immutable i, ref e; es)
>  {
>  moveEmplace(e, _store[_length + i]); // TODO remove
> `move` when compiler does it for us
>  }
>  _length = cast(Length)(_length + Es.length); // TODO
> better?
>  }
>
> Is the usage of `enforce` to check for out of bounds (fullness)
> idiomatic D or should an `assert()` be used instead?

It depends entirely on how you intend for it to be used. Is it a bug if
anyone attempts to append when there's no space, or is that normal program
behavior that should be handled and recovered from? Most typically, with D,
it would be considered a bug, and the caller should be sure that there's
room before attempting to append, but depending on your use case, using
exceptions may be more appropriate. However, if you don't have a really good
reason to treat it as a recoverable error condition rather than a bug, I'd
suggest that you treat it as a bug.

Also, you could treat it as a RangeError instead of an AssertError (the main
difference being that with assert, the checks go away when -release is
used). However, if your code would trigger a RangeError when accessing the
static array anyway, then you could just forgo all checks entirely and let
druntime throw a RangeError.

- Jonathan M Davis






Re: Struct initialization has no effect or error?

2019-10-03 Thread Jonathan M Davis via Digitalmars-d-learn
On Wednesday, October 2, 2019 11:48:46 PM MDT Mike Parker via Digitalmars-d-
learn wrote:
> On Thursday, 3 October 2019 at 04:57:44 UTC, mipri wrote:
> > On Thursday, 3 October 2019 at 04:33:26 UTC, Brett wrote:
> >> I was trying to avoid such things since X is quite long in
> >> name. Not a huge deal... and I do not like the syntax because
> >> it looks like a constructor call.
> >
> > It is a constructor call, though. You can define your own as
>
> > well:
> Technically it's a struct literal. It's only a constructor if you
> define one, in which case struct literals no longer work. E.g.,
>
> struct Foo {
>  int x;
>
>  this(int a, int b) { x = a + b; }
> }
>
> Without the constructor, the literal Foo(10) would be valid, but
> with the constructor you'll get a compiler error.

Yeah. Syntactically, there's no real distinction, but the compiler doesn't
do something like generate a constructor for you, and unlike with a
constructor, you won't necessarily get errors if you do something like use
too few arguments. So, if you had

struct S
{
int x;
}

auto s = S(42);

and then changed it to

struct S
{
int x;
int y;
}

auto s = S(42);

the code would continue to compile without complaint. And if you had
something like

struct S
{
string s;
int i;
}

auto s = S("hello", 42);

struct S
{
string s;
int foo;
int i;
}

auto s = S("hello", 42);

you end up initializing the wrong members. The same if had

struct S
{
int x;
int y;
}

auto s = S(12, 99);

and changed it to

struct S
{
int y;
int x;
}

auto s = S(12, 99);

The fact that struct literals exist basically forces you to declare
constructors if you don't want to have to worry about breaking code by
rearranging member variables.

Personally, I think that using struct literals is just begging for bugs in
your code, so I never use them, and I always declare constructors. I wish
that struct literals weren't a thing at all, but some folks clearly like
them.

If using a struct literal with braces without providing the member names
gets deprecated like apparently Walter wants to do, then maybe using the
construction syntax for struct literals would be deprecated as well, which
would at least improve the situation. struct literals with member names are
still error-prone, but at least you then eliminate the bugs where you
initialize the wrong members and instead just get the ones where new members
end up with the default value whether it's appropriate or not.

- Jonathan M Davis





Re: C++ base constructor call vs. D's

2019-10-02 Thread Jonathan M Davis via Digitalmars-d-learn
On Wednesday, October 2, 2019 11:22:40 AM MDT Just Dave via Digitalmars-d-
learn wrote:
> I was reading the C++ to D page, and came across this little bit
> about when to call the base class constructor:
>
> "It's superior to C++ in that the base constructor call can be
> flexibly placed anywhere in the derived constructor."
>
> Isn't there some inherent danger of not calling the base
> constructor first? Wouldn't C++'s method actually be equal in the
> effect that you could just overwrite whatever value the base
> class set in the derived class?

The key difference in D has to do with init values. With a struct, the
values that its members are directly initialized with create its init value.
So,

struct S
{
string s = "hello";
int i;
int j = 42;
}

results in

assert(S.init == S(s, 0, 42));

For classes, because the type system always treats them as references, the
init value is null. So

class C
{
string s = "hello";
int i;
int j = 42;
}

results in

assert(C.init is null);

However, the class still has the equivalent of the struct's init value - you
just can't access it directly, and it's not used for default initialization,
just in construction. So, before a class' constructor is run, its memory is
initialized with that hidden init value, meaning that you don't have garbage
when you enter the constructor, and the class is already fully formed in
terms of its type and virtual table before any constructors are called,
whereas in C++, if you're in a base class constructor, that class object is
not yet the derived class type. So, it's actually possible and safe to call
virtual functions from within a class constructor in D, whereas it isn't in
C++. So, this example prints "D"

class C
{
string foo()
{
return "C";
}

this()
{
import std.stdio;
writeln(foo());
}
}

class D : C
{
override string foo()
{
return "D";
}
}

void main()
{
auto d = new D;
}

whereas C++ equivalent would likely blow up in your face.

- Jonathan M Davis





Re: Why dynamic array is InputRange but static array not.

2019-09-24 Thread Jonathan M Davis via Digitalmars-d-learn
On Tuesday, September 24, 2019 1:35:24 AM MDT lili via Digitalmars-d-learn 
wrote:
> Hi:
>in phobos/std/range/primitives.d has this code
>   ```
> static assert( isInputRange!(int[]));
> static assert( isInputRange!(char[]));
> static assert(!isInputRange!(char[4]));
> static assert( isInputRange!(inout(int)[]));
>  ```
>   but the dynamic array and static array neither not has
> popFront/front/empty.
> https://dlang.org/spec/arrays.html#array-properties properties

Because for something to be a range, it must be possible for it to shrink.
popFront works with a dynamic array, because dynamicy arrays have a dynamic
size. It's basically just

void popFront(T)(ref T[] a)
{
a = a[1 .. $];
}

However, static arrays have a fixed size, so it's not possible to implement
popFront for them. If you want to use a static array as a range, then you
need to slice it to get a dynamic array - though when you do that, make sure
that the dynamic array is not around longer than the static array, because
it's just a slice of the static array, and if the static array goes out of
scope, then the dynamic array will then refer to invalid memory.

- Jonathan M Davis





Re: wstring comparison is failing

2019-09-23 Thread Jonathan M Davis via Digitalmars-d-learn
On Monday, September 23, 2019 5:22:14 PM MDT Brett via Digitalmars-d-learn 
wrote:
> On Monday, 23 September 2019 at 20:45:00 UTC, destructionator
>
> wrote:
> > On Mon, Sep 23, 2019 at 08:38:03PM +, Brett via
> >
> > Digitalmars-d-learn wrote:
> >> The only thing is that szExeFile is a static
> >> wchar array... which shouldn't effect the comparison.
> >
> > are you sure about that?
> >
> > with a static size, the lengths of the two sides won't match...
>
> I guess you are probably right... I was thinking that it would
> compare up to a null terminator. Seems kinda buggy... maybe the
> compiler needs to give a warning? After all, compared a fixed
> size array with a dynamic array then will almost always fail
> since it is unlikely the sizes will match...
>
> ...rather than failing silently.

D does virtually nothing with null terminators. String literals have them
one index past their end so that you can pass them to C functions without D
code actually seeing the null terminator, but otherwise, you have to add the
null terminator when passing to C code (e.g. with toStringz or toUTFz), and
you have to strip it off when getting a string from C code (e.g. with
fromStringz). Other than functions specifically designed to convert to and
from C strings, D code is going to treat null terminators just like any
other character, because D strings are not null-terminated.

- Jonathan M Davis





Re: what is the mean that call function start with a dot.

2019-09-22 Thread Jonathan M Davis via Digitalmars-d-learn
On Saturday, September 21, 2019 10:32:08 PM MDT Paul Backus via Digitalmars-
d-learn wrote:
> On Sunday, 22 September 2019 at 04:15:53 UTC, lili wrote:
> > Hi:
> >
> >yesterday I saw some d code, where is an .fn() call syntax,
> >
> > what is it mean.
>
> It means that `fn` is looked up in the module's top-level scope.
>
> Source: https://dlang.org/spec/expression.html#identifier

which includes any top-level imports, so the symbol in question isn't
necessarily within the module. The key thing is that when a symbol is
preceded by a dot, no local or member symbols are taken into account. It
makes it so that if you have a local or member symbol which has the same
name as a top-level symbol, you're able to directly reference the top-level
symbol without providing the entire module path.

So, for instance, if a socket class had a close method, that close method
could call the C function, close, with .close, whereas otherwise, it would
have to do something like core.sys.posix.unistd.close (at least on POSIX
systems) to call the C function, since if it called close without any kind
of path, it would end up recursively calling the close member function,
because the module-level function is shadowed by the member function.

- Jonathan M Davis






Re: Looking for a Simple Doubly Linked List Implementation

2019-09-21 Thread Jonathan M Davis via Digitalmars-d-learn
On Saturday, September 21, 2019 12:52:23 PM MDT Dennis via Digitalmars-d-
learn wrote:
> Since I marked the method as const, `auto a = head` got the type
> const(Node!T) and `a = a.next` no longer compiled. With structs
> you can declare a const(Node!T)* (mutable pointer to const node),
> but I don't know if I can declare a mutable reference to a const
> class, so I switched to structs.

You have to use std.typecons.Rebindable if you want to have the equivalent
of const(T)* for class references, because the type system doesn't
distinguish between a class and a reference to a class. As it is, Rebindable
is pretty much a hack that's questionably legal per the type system (but
it's in Phobos, so I'm sure that it will continue to work). Ideally, there
would be a way to do it in the language, but the assumptions that the
compiler currently makes when dealing with classes makes that difficult.

In general though, if you're not going to use inheritance, then there isn't
much point in using a class instead of a struct unless you want to force it
to live on the heap (and that only really matters if you're dealing with
something that's publicly available for others to muck with, whereas nodes
in a linked list are normally private to the list, so it's easy to ensure
that they're only ever on the heap even if they're structs).

- Jonathan M Davis





Re: Why must a bidirectional range also be a forward range?

2019-09-21 Thread Jonathan M Davis via Digitalmars-d-learn
On Friday, September 20, 2019 7:08:03 AM MDT Joseph Rushton Wakeling via 
Digitalmars-d-learn wrote:
> On Thursday, 19 September 2019 at 22:55:55 UTC, Jonathan M Davis
>
> wrote:
> > For better or worse, ranges were more or less set up as a
> > linear hierarchy, and it's unlikely that use cases for
> > bidirectional ranges which aren't forward ranges are common. I
> > expect that it's a bit like infinite, bidirectional ranges. In
> > theory, they could be a thing, but the use cases for them are
> > uncommon enough that we don't really support them. Also, I
> > expect that most range-based algorithms which operate on
> > bidirectional ranges would require save anyway. A lot of
> > algorithms do to the point that basic input ranges can be
> > incredibly frustrating to deal with.
> >
> > [ ... ]
>
> Thanks for the characteristically thorough description of both
> the design considerations and the history involved.
>
> On reflection it occurs to me that the problem in my thinking may
> be the idea that `save` should result in a full deep copy.  If
> instead we go by how `save` is implemented for dynamic arrays,
> it's only ever a shallow copy: it's not possible to make valid
> assumptions of reproducible behaviour if the original copy is
> modified in any way.
>
> If instead we assume that `save` is only suitable for temporary
> shallow-copies that are made under the hood of algorithms, then
> my problems go away.

save is supposed to result copies that can be independently iterated over.
So, code such as

foreach(r; range.save)
{}
auto arr = r.save.array();
assert(equal(arr, r));

should work. How that's implemented underneath the hood doesn't really
matter. However, none of that really takes into account mutation of the
elements. The range API pretty much assumes that you don't ever modify any
of the elements in the range as you iterate over them. So, if you do
something like

auto orig = range.save;
range.front = 42;

whether orig.front is then 42 is implementation-dependent. So, if you're
modifying elements as you go, then the behavior you get is going to be
highly dependent on what you're doing with the ranges, and certainly, if
you're only using a range within a very specific context, it can be
implemented in a way that works in that context but doesn't work with
range-based functions in general. You just run the risk of problems if you
then later modify the code to use other range-based functions which don't
necessarily work with whatever you've done with the range.

As for temporary, shallow copies, IIRC, isForwardRange requires that save
returns exactly the same type as the original. So, while you can certainly
have a range referring to a data structure without owning any of the data
(after all, that's what happens with dynamic arrays), you can't have a range
of one type that owns the data and then have save return a range type which
just refers to the data unless the range is written in a way that you can
have both within the same type.

One example of avoiding the need to deep-copy with save where one range is
at least sort of the owner whereas the others aren't is how dxml's stax
parser works. The ranges share a context that keeps track of how far into
the range the farthest range is, and popFront only does the validation when
the range that popFront is being called on is the farthest of any range.
That way, the stuff related to validating end tags didn't need to be
deep-copied, but save always returns exactly the same type, and you get
exactly the same behavior regardless of which range gets iterated farther
first (or even if one is iterated farther and then another is iterated
beyond it).

If popFront every throws, then that range becomes invalid, but the others
are fine. The validation other than that for matching end tags currently all
gets done every time, so all ranges would throw in the same place for errors
other than end tags that don't match, but the same is also true for when the
end tags don't match, because even though that validation is only done for
the farthest range, if it fails, the shared context is left in exactly the
same state, and any other ranges that reach that point would then throw like
the first range did.

Without realizing that the validation for the end tags didn't have to be
done for every instance of the range but only the one which was farthest
along, I would have been screwed with regards to save, because deep-copying
would have been required. I'm not sure that that particular trick is widely
applicable, but it is an example of how save can do something other than a
deep copy even though having each range do exactly the same work would have
required a deep copy.

> > Assuming we were redesigning the range API (which may happen if
> > we do indeed end up doing a Phobos v2), then maybe we could
> > make it so that bidirectional ranges don't have to be forward
> > ranges, but honestly _any_ ranges which aren't forward ranges
> > are a bit of a problem. We do need to 

Re: Avoid gratuitous closure allocations

2019-09-20 Thread Jonathan M Davis via Digitalmars-d-learn
On Friday, September 20, 2019 5:21:22 AM MDT Ali Çehreli via Digitalmars-d-
learn wrote:
> tl;dr Instead of returning an object that uses local state, return an
> object that uses member variables.

The other issue this helps with is problems related to having multiple
contexts. IIRC, without it, some predicates don't work due to the compiler
complaining about there being more than one context. However, I pretty much
always use static structs for ranges though, so I haven't run into the issue
recently enough to recall the exact details. In general though, I'd say that
it's best pratice to use static structs in functions and avoid needing local
context. Sometimes, it makes sense to do so, but in general, giving your
struct access to the local scope in the function is going to result in
closures being allocated when they could have easily been avoided.

- Jonathan M Davis






Re: Why must a bidirectional range also be a forward range?

2019-09-19 Thread Jonathan M Davis via Digitalmars-d-learn
On Thursday, September 19, 2019 3:31:32 AM MDT Joseph Rushton Wakeling via 
Digitalmars-d-learn wrote:
> Hello folks,
>
> A question that occurred to me while implementing a new data
> structure recently, which I'm not sure I've ever seen a reason
> for.
>
> Why must bidirectional ranges also be forward ranges (as opposed
> to just input ranges)?
>
> It doesn't seem to me that the `save` property is inherently
> required to iterate backwards over a range -- just the `back` and
> `popBack` methods.
>
> It makes sense that, for bidirectionality, the range needs to be
> deterministic, so that iterating backward gives the exact same
> elements as iterating forward, just in reverse order.  But it
> seems strange to require the `save` property in order to
> automatically assume deterministic behaviour.
>
> For context, the use-case I have is a data structure which stores
> an internal buffer as an array.  A robust `save` method would
> therefore have to duplicate the array (or at least the active
> subset of its contents).  This means a fresh heap allocation per
> `save`, which has some nasty implications for phobos algorithms
> that eagerly `.save` when they can.
>
> So, I'd rather not implement `save` in this case.  But there is
> nothing that blocks implementing `back` and `popBack`; yet I
> can't use these with any of the functionality that requires
> bidirectionality, because the current `isBidirectionalRange`
> check requires `save`.
>
> So what gives?  Are there some reasons for the `save` requirement
> on bidirectional ranges that I'm missing?  And regardless, any
> advice on how to handle my particular use-case?
>
> Thanks & best wishes,
>
>-- Joe

For better or worse, ranges were more or less set up as a linear hierarchy,
and it's unlikely that use cases for bidirectional ranges which aren't
forward ranges are common. I expect that it's a bit like infinite,
bidirectional ranges. In theory, they could be a thing, but the use cases
for them are uncommon enough that we don't really support them. Also, I
expect that most range-based algorithms which operate on bidirectional
ranges would require save anyway. A lot of algorithms do to the point that
basic input ranges can be incredibly frustrating to deal with.

Assuming we were redesigning the range API (which may happen if we do indeed
end up doing a Phobos v2), then maybe we could make it so that bidirectional
ranges don't have to be forward ranges, but honestly _any_ ranges which
aren't forward ranges are a bit of a problem. We do need to support them on
some level for exactly the kind of reasons that you're looking to avoid save
with a bidirectional range, but the semantic differences between what makes
sense for a basic input range and a forward range really aren't the same (in
particular, it works far better for basic input ranges to be reference
types, whereas it works better for forward ranges to be value types).

As it stands, I don't think that we can change isBidirectionalRange, because
it's likely that most code using it relies on its check for isForwardRange.
So, I think that we're stuck for the moment, but it is food for thought in a
possible range API redesign. I'll add it to my notes on the topic. Some
aspects of a range API redesign should look like are pretty clear at this
point, whereas others are very much an open question.

Ideally, I'd like to force basic input ranges to be reference types, and
forward ranges to be value types, but I'm not sure that that's reasonable in
practice. It would really clean up some of the semantics of ranges, but it
would also likely require allocating a lot more stuff on the heap than would
be desirable. Either way, having bidirectional ranges not need to have the
equivalent of save would mean treating them as more of an add-on capability
(like length) rather than having the kind of hierarchy that we have now. I
don't know if that's ultimately a good or a bad thing.

- Jonathan M Davis





Re: Deprecation message sources

2019-09-17 Thread Jonathan M Davis via Digitalmars-d-learn
On Tuesday, September 17, 2019 2:34:00 PM MDT Steven Schveighoffer via 
Digitalmars-d-learn wrote:
> On 9/17/19 4:16 PM, Anonymouse wrote:
> > On Tuesday, 17 September 2019 at 19:31:53 UTC, Steven Schveighoffer 
wrote:
> >> I'd hate to say the answer is to special case Nullable for so many
> >> functions, but what other alternative is there?
> >>
> >> -Steve
> >
> > Nullable isn't alone, std.json.JSONType causes a literal wall of text of
> > deprecation warnings.
> >
> > import std.stdio;
> > import std.json;
> >
> > void main()
> > {
> >
> >  writeln(JSONValue.init.type);
> >
> > }
> >
> > https://run.dlang.io/is/J0UDay
>
> I mean, I'm OK with the idea, but having these deprecation messages is
> helping nobody. I can't figure out if there's something I'm supposed to,
> or can, change in order to get rid of them.
>
> There are quite a few places where it is flagging my code for Nullable
> usage without get, and I'm fixing those. But I'll still be left with
> this mess of deprecation messages from Phobos and vibe.d. I don't even
> know where or if it will break once the alias this is removed.

I ran into problems along those lines with dxml recently, and I couldn't
figure out why one of the deprecation messages was being triggered. It
seemed to have to do with isInputRange, but I couldn't figure out where in
my code was resulting in that problem. I tried to track it down by compiling
Phobos with the alias this outright removed from Nullable (with the idea
that I'd then hopefully get some decent error messages wherever the real
problem was), and dxml's tests then compiled and ran just fine with no
deprecation messages. So, I don't know what to do about it. I suspect that
deprecation messages are being triggered simply by code trying to use
Nullable in template constraints rather than just when it's actually used in
proper code, but I don't know.

I ran into problems along those lines when I last tried to deprecate
TickDuration (which is why it's not yet deprecated). Template constraints
were triggering deprecation messages when I didn't think that they should
be, but unfortunately, I didn't have time to narrow down what was going on
so that I could create a proper bug report for it, and I haven't gotten back
to it.

- Jonathan M Davis





Re: Deprecation message sources

2019-09-17 Thread Jonathan M Davis via Digitalmars-d-learn
On Tuesday, September 17, 2019 5:28:33 PM MDT H. S. Teoh via Digitalmars-d-
learn wrote:
> On Tue, Sep 17, 2019 at 08:55:27PM +, Johan Engelen via
> Digitalmars-d-learn wrote: [...]
>
> > Wow. How come this is not caught by the CI testing?
>
> [...]
>
> Is the CI setup to detect deprecations and flag them as failures?
>
> It's either that, or many cases are not tested because Phobos has a lot
> of templates, not all of which are instantiated with the specific
> combination of template arguments that triggers deprecation messages.

Yes. Seb made druntime and Phobos compile with -de a while back in order to
make sure that all cases where deprecations pop up actually get fixed - but
of course, that's only going to catch the cases that are in the tests, and
it probably wouldn't be hard to trigger a bunch of deprecations in Phobos by
doing something like using a range of Nullable, since implicit conversions
would probably get triggered all over the place - especially if it's the
case that the deprecation message gets triggered by stuff like static if
tests and template constraints (I'm not sure that it does in this case, but
I have seen problems in the past where template constraints triggered
deprecation messages; last time I tried to deprecate TickDuration, I ran
into a bunch of problems like that, which is why it hasn't been deprecated
yet).

- Jonathan M Davis





Re: Slicing upward

2019-09-14 Thread Jonathan M Davis via Digitalmars-d-learn
On Saturday, September 14, 2019 5:34:35 AM MDT Brett via Digitalmars-d-learn 
wrote:
> I have an algorithm that is most efficiently implement by taking
> an array and slicing it upward, meaning removing the leading
> elements.
>
> Because the algorithm is complex(deterministic but chaotic) and
> deals with multiple arrays it is difficult to efficiently use
> slicing.
>
> Is there some easy way to take an array and slice it in a way
> that as the array grows any slices will "shift".
>
> Essentially it is sort of reversing how normal slices are
> represented
>
> [2,4,1,64]
> slice last 2, [1,64]
>
> append 3 to either one and both become
>
> [2,4,1,64,3]
> [1,64,3]
>
>
> In my case I will never have any slices in the middle.
>
> The easiest way, at the cost of size_t is to have an indexer that
> tells one how much to offset in to a standard array, basically
> skip the head.
>
> In the first it is 0 and the second it is 2.
>
> Since I already have the array though it is wasting space so I'm
> wondering if there is an easier way.
>
>
> Reversing the direction of the arrays does notwork because this
> then require massive copying of the array to prepend values.
>
> One can make an array type emulating this behavior but I'd rather
> have a simpler solution that is inline with slices, Else I'll
> just use the indexer method for simplicity.

No, that fundamentally goes against how D's dynamic arrays work. Their
representation is essentially

struct DynamicArray(T)
{
size_t length;
T* ptr;
}

They don't even have a concept of ownership. They merely point to a slice of
memory, and that memory isn't necessarily GC-allocated. In order to append
to a dynamic array, the GC looks at it to see if it's a slice of a block of
memory that the GC has allocated for dynamic arrays. If it is such a block
of memory _and_ the metadata associated with that block of memory indicates
that no other dynamic arrays have ever been expanded into the memory beyond
that dynamic array, then the GC will put the element being appended into the
next spot after the end of the dynamic array, adjust the metadata, and then
increment the length member of the dynamic array that it was passed. If, on
the other hand, that block of memory was not allocated by the GC, was
allocated by the GC for something other than dynamic arrays, or if the
metadata indicates that a dynamic array has already grown into the space
beyond the end of that dynamic array, then it will have to allocate a new
block of memory, copy the elements to that new block of memory, and then
adjust the dynamic array that it was passed so that its ptr and length
members then refer to the new block of memory.

In the case where no allocation occurs, in order for any other dynamic
arrays which happen to refer to that same block of memory and which happen
to have their last element at the same spot as the dynamic array being
appended to then also include that new element, the GC would have to do a
sweep of all of the memory in the program to see if any other dynamic arrays
exist which are slices of that same block of memory and adjust them
(basically, it would have to do what it does when it does a collection when
determining whether a block of memory can be freed). The same would be the
case if the dynamic array had to be reallocated, only then the ptr member
would have to be adjusted as well. Not only would such an approach be
horribly inefficient, but it wouldn't work with all of the use cases where
you want the other dynamic arrays which are slices of that same block of
memory to continue to refer to the exact same piece of that block of memory
that they've been referring to.

The entire design of D's dynamic arrays is built around the idea that they
don't own or manage any memory at all. They are merely slices of it. What
you're looking for implies an ownership relationship which simply does not
exist with D's dynamic arrays.

With D's dynamic arrays, there is no concept of there being a main one which
others are slices of. If two dynamic arrays are slices of the same block of
memory (whether they refer to exactly the same part of it or not), then they
are equal in standing. Neither of them owns the memory. They just point to
it.

To do what you're looking to do with dynamic arrays, you'd need another
layer of indirection - e.g. T[]* - where you then had a separate data type
for your "slices" of that dynamic array. e.g.

struct Slice(T)
{
T[]* arr;
size_t startIndex;
size_t length;
}

would allow you to have a "slice" which always referred to the same elements
of a dynamic array even if the dynamic array were reallocated, or

struct Slice(T)
{
T[]* arr;
size_t startIndex;
}

would allow you to have the "slice" to expand when the dynamic array it
refers to expands.

In either case, if you're using a dynamic array to store the data, you'd
have to make sure that it was somewhere where it was going to stay valid as
long as the "slices" were around (e.g. a static 

Re: Should an 'extern(C++, "ns"):' override previous ones in the same scope?

2019-09-08 Thread Jonathan M Davis via Digitalmars-d-learn
On Sunday, September 8, 2019 12:12:53 PM MDT Exil via Digitalmars-d-learn 
wrote:
> On Saturday, 7 September 2019 at 22:19:48 UTC, Jonathan M Davis
>
> wrote:
> > On Saturday, September 7, 2019 3:40:58 PM MDT Exil via
> >
> > Digitalmars-d-learn wrote:
> >> On Saturday, 7 September 2019 at 17:22:07 UTC, Jonathan M Davis
> >>
> >> wrote:
> >> > @safe:
> >> > @system:
> >> >
> >> > then @system overrides @safe.
> >>
> >> Just to add onto this, you can't do:
> >>  @safe @system void foo(); // error
> >>
> >> but you can do:
> >>  extern(C++, ns1) extern(C++, ns2) void foo(); // ok
> >
> > It makes no sense to apply multiple namespaces to the same
> > symbol. I expect that this behavior is due to a lack of testing
> > (the same with the out of order weirdness in the other post).
> > It's the sort of thing that you test when you're trying to make
> > sure that the feature does the right thing when people use it
> > incorrectly, not the sort of thing when you're trying to make
> > sure that the feature works as intended, so it's easy to forget.
> >
> > My guess is that this behavior leaked its way in due to the
> > fact that you
> > need to be able to put multiple extern(C++) declarations on a
> > symbol when
> > you use extern(C++, struct) or extern(C++, class) in addition
> > to the
> > extern(C++) for the namespace.
> >
> > - Jonathan M Davis
>
> You don't need to make guesses or assumptions. It most definitely
> was intentional.
>
> https://github.com/dlang/dmd/commit/4b2578e208f2af9a02159fc2d8d87fb17b0900
> 5e#diff-62dcb5f0ffc3089b7565897d8beb3322R617
>
>
> By the looks of it, this feature was also implemented before
> extern(C++, struct/class).

Well, it's inconsistent with the rest of the language and bad design IMHO.
And even if it were desirable behavior, it clearly becomes a mess with the
ordering once the attributes are no longer directly on the symbol. Of
course, ideally, the whole extern(C++) feature with identifiers instead of
strings for the namespace would be deprecated anyway.

- Jonathan M Davis





Re: Should an 'extern(C++, "ns"):' override previous ones in the same scope?

2019-09-08 Thread Jonathan M Davis via Digitalmars-d-learn
On Sunday, September 8, 2019 3:03:31 AM MDT Max Samukha via Digitalmars-d-
learn wrote:
> On Saturday, 7 September 2019 at 22:19:48 UTC, Jonathan M Davis
>
> wrote:
> > On Saturday, September 7, 2019 3:40:58 PM MDT Exil via
> >
> > Digitalmars-d-learn wrote:
> >> On Saturday, 7 September 2019 at 17:22:07 UTC, Jonathan M Davis
> >>
> >> wrote:
> >> > @safe:
> >> > @system:
> >> >
> >> > then @system overrides @safe.
> >>
> >> Just to add onto this, you can't do:
> >>  @safe @system void foo(); // error
> >>
> >> but you can do:
> >>  extern(C++, ns1) extern(C++, ns2) void foo(); // ok
> >
> > It makes no sense to apply multiple namespaces to the same
> > symbol. I expect that this behavior is due to a lack of testing
> > (the same with the out of order weirdness in the other post).
> > It's the sort of thing that you test when you're trying to make
> > sure that the feature does the right thing when people use it
> > incorrectly, not the sort of thing when you're trying to make
> > sure that the feature works as intended, so it's easy to forget.
> >
> > My guess is that this behavior leaked its way in due to the
> > fact that you
> > need to be able to put multiple extern(C++) declarations on a
> > symbol when
> > you use extern(C++, struct) or extern(C++, class) in addition
> > to the
> > extern(C++) for the namespace.
> >
> > - Jonathan M Davis
>
> I wonder how that undocumented and not well thought-through (or
> having some unobvious justifications) feature made it into a
> stable release.
>
> Anyway, thank you for your help. I will probably file a bug
> report when I have time.

The C++ support has been improved kind of piecemeal over time, and
initially, none of it was documented. So, it wasn't exactly fully planned
out when it was added. IIRC, it was added originally so that the dmd
frontend could be moved to D. The DIP process was very different at that
point, and it was much more common then for Walter or one of the other core
developers to just propose a feature in a PR and get it merged. I expect
that the oddities in the implementation stem from stuff that whoever
implemented it didn't think to try. The whole process is much more rigorous
now than it used to be.

- Jonathan M Davis





  1   2   3   4   5   6   7   8   9   10   >