Re: Splitting up large dirty file
On Monday, 21 May 2018 at 15:00:09 UTC, Dennis wrote: I want to be convinced that Range programming works like a charm, but the procedural approaches remain more flexible (and faster too) it seems. Thanks for the example. On Monday, 21 May 2018 at 22:11:42 UTC, Dennis wrote: In this case I used drop to drop lines, not characters. The exception was thrown by the joiner it turns out. ... From the benchmarking I did, I found that ranges are easily an order of magnitude slower even with compiler optimizations: My general experience is that range programming works quite well. It's especially useful when used to do lazy processing and as a result minimize memory allocations. I've gotten quite good performance with these techniques (see my DConf talk slides: https://dconf.org/2018/talks/degenhardt.html). Your benchmarks are not against the file split case, but if you benchmarked that you may have also seen it as slow. It that case you may be hitting specific areas where there are opportunities for performance improvement in the standard library. One is that joiner is slow (PR: https://github.com/dlang/phobos/pull/6492). Another is that the write[fln] routines are much faster when operating on a single large object than many small objects. e.g. It's faster to call write[fln] with an array of 100 characters than: (a) calling it 100 times with one character; (b) calling it once, with 100 characters as individual arguments (template form); (c) calling it once with range of 100 characters, each processed one at a time. When joiner is used as in your example, you not only hit the joiner performance issue, but the write[fln] issue. This is due to something that may not be obvious at first: When joiner is used to concatenate arrays or ranges, it flattens out the array/range into a single range of elements. So, rather than writing a line at a time, you example is effectively passing a character at a time to write[fln]. So, in the file split case, using byLine in an imperative fashion as in my example will have the effect of passing a full line at a time to write[fln], rather than individual characters. Mine will be faster, but not because it's imperative. The same thing could be achieved procedurally. Regarding the benchmark programs you showed - This is very interesting. It would certainly be worth additional looks into this. One thing I wonder is if the performance penalty may be due to a lack of inlining due to crossing library boundaries. The imperative versions aren't crossing these boundaries. If you're willing, you could try adding LDC's LTO options and see what happens. There are some instructions in the release notes for LDC 1.9.0 (https://github.com/ldc-developers/ldc/releases). Make sure you use the form that includes druntime and phobos. --Jon
Re: C style callbacks fix for member callbacks
On Monday, 21 May 2018 at 02:23:27 UTC, ag0aep6g wrote: I tried this. Your code crashes in windows dmd x86 x64. Hm. Works for me in a virtual machine. But I'm not surprised that it's fragile. It might be completely wrong, and it just happens to look alright on my machine. https://run.dlang.io/is/CMNnJY Shows the static version produces the same code as the non-static. The code was probably compiled on linux. I can't tell though on my machine what is going on since I cannot disassemble properly and what I do see is far more complex.
Re: Splitting up large dirty file
On Monday, 21 May 2018 at 17:42:19 UTC, Jonathan M Davis wrote: On Monday, May 21, 2018 15:00:09 Dennis via Digitalmars-d-learn wrote: drop is range-based, so if you give it a string, it's going to decode because of the whole auto-decoding mess with std.range.primitives.front and popFront. In this case I used drop to drop lines, not characters. The exception was thrown by the joiner it turns out. On Monday, 21 May 2018 at 17:42:19 UTC, Jonathan M Davis wrote: I find Exceptions in range code hard to interpret. Well, if you just look at the stack trace, it should tell you. I don't see why ranges would be any worse than any other code except for maybe the fact that it's typical to chain a lot of calls, and you frequently end up with wrapper types in the stack trace that you're not necessarily familiar with. Exactly that: stack trace full of weird mangled names of template functions, lambdas etc. And because of lazy evaluation and chains of range functions, the line number doesn't easily show who the culprit is. On Monday, 21 May 2018 at 17:42:19 UTC, Jonathan M Davis wrote: In many cases, ranges will be pretty much the same as writing loops, and in others, the abstraction is worth the cost. From the benchmarking I did, I found that ranges are easily an order of magnitude slower even with compiler optimizations: https://run.dlang.io/gist/5f243ca5ba80d958c0bc16d5b73f2934?compiler=ldc=-O3%20-release ``` LDC -O3 -release Range Procedural Stringtest: ["267ns", "11ns"] Numbertest: ["393ns", "153ns"] DMD -O -inline -release Range Procedural Stringtest: ["329ns", "8ns"] Numbertest: ["1237ns", "282ns"] ``` This first range test is an opcode scanner I wrote for an assembler. The range code is very nice and it works, but it needlessly allocates a new string. So I switched to a procedural version, which runs (and compiles) faster. This procedural version did have some bugs initially though. The second test is a simple number calculation. I thought that the range code inlines to roughly the same procedural code so it could be optimized the same, but there remains a factor 2 gap. I don't know where the difficulty is, but I did notice that switching the maximum number from int to enum makes the procedural version 0 ns (calculated at compile time) while LDC can't deduce the outcome in the range version (which still runs for >300 ns).
Re: scope guards & debugger breakpoints?
On 5/21/18 1:50 PM, Robert M. Münch wrote:
On 2018-05-21 17:24:12 +, Steven Schveighoffer said:
I'm not 100% sure but I expect:
scope(failure)
someCode();
putting a breakpoint on someCode should work.
When calling a function an then setting the breakpoint there, like in
someCode() yes, that should work.
I used code like this:
scope(failure) {
... several lines of code
}
And for this it seems (maybe I need to check in more detail) that in
MSVC, I can set a breakpoint, but the breakpoint is not triggered when
an exception is thrown.
Seems like the debugger not understanding D source in that case.
What you could try is:
void breakOnThis()
{
return; // breakpoint here
}
scope(failure)
{
breakOnThis();
...
}
and then see where it goes when you step out of that function.
-Steve
Re: Temporary file creation for unittests
On Monday, 21 May 2018 at 15:20:14 UTC, Dr.No wrote:
On Monday, 21 May 2018 at 15:16:11 UTC, Atila Neves wrote:
On Friday, 18 May 2018 at 15:16:52 UTC, Russel Winder wrote:
Hi,
What's the current official position on how to create
temporary files for use during a unittest. I found
Not official, but...
import unit_threaded;
with(const Sandbox()) {
writeFile("myfile.txt", "contents");
shouldExist("myfile.txt");
shouldEqualContent("myfile.txt", "contents");
fileShouldContain("myfile.txt", "cont");
}
Atila
I've never seen "should" being in used in function names
before...
There's a whole lot of them here:
https://github.com/atilaneves/unit-threaded/blob/master/source/unit_threaded/should.d
Re: Temporary file creation for unittests
On Monday, 21 May 2018 at 17:03:40 UTC, Russel Winder wrote:
On Mon, 2018-05-21 at 15:16 +, Atila Neves via
Digitalmars-d-learn wrote:
On Friday, 18 May 2018 at 15:16:52 UTC, Russel Winder wrote:
> Hi,
>
> What's the current official position on how to create
> temporary files for use during a unittest. I found
Not official, but...
import unit_threaded;
with(const Sandbox()) {
writeFile("myfile.txt", "contents");
shouldExist("myfile.txt");
shouldEqualContent("myfile.txt", "contents");
fileShouldContain("myfile.txt", "cont");
}
Atila
OK, we like this. A lot.
:)
Given I use Unit-Threaded, why did I not know this. Ah, OK,
RTFM. :-)
It's got so many features that I don't know how to document
everything and make it accessible at the same time.
Did I mention how much I like this RAII approach?
Me too - RAII is definitely C++'s gift to the world. I've been
abusing `with` lately quite a bit. I think it's underused.
Re: auto & class members
On Monday, May 21, 2018 16:05:00 Steven Schveighoffer via Digitalmars-d- learn wrote: > On 5/21/18 3:22 PM, Jonathan M Davis wrote: > > That's basically what I was suggesting that he do, but I guess that I > > wasn't clear enough. > > Well one thing that seems clear from this example -- we now have > __traits(isSame) to tell if lambdas are the same, but it looks like the > compiler doesn't subscribe to that belief... > > https://run.dlang.io/is/FW3mVq > > We should fix that... Yeah. That part of lambdas has always been a problem. My guess here is that the problem stems from the fact that they're declared in separate scopes, but I don't know. Regardless of the reason for the failure though, that example really needs to work, or most anything that cares about lambdas being the same is going to have problems. - Jonathan M Davis
Re: auto & class members
On 5/21/18 3:22 PM, Jonathan M Davis wrote: That's basically what I was suggesting that he do, but I guess that I wasn't clear enough. Well one thing that seems clear from this example -- we now have __traits(isSame) to tell if lambdas are the same, but it looks like the compiler doesn't subscribe to that belief... https://run.dlang.io/is/FW3mVq We should fix that... -Steve
Re: is ==
On 5/21/18 3:20 PM, Jonathan M Davis wrote:
On Monday, May 21, 2018 14:40:24 Steven Schveighoffer via Digitalmars-d-
learn wrote:
For me, the code smell is using arr is null (is it really necessary to
check for a null pointer here?), for which I always have to look at more
context to see if it's *really* right.
Really? I would never expect anyone to use is unless they really cared about
whether array was null. I'd be concerned about whether the code in general
was right, because treating null as special gets tricky, but that particular
line wouldn't concern me.
Don't get me wrong, they probably *do* mean to check if it's null. But
do they *need* to check?
I'll borrow from your example below:
if(arr != null && arr == arr2)
Replace this with:
if(arr !is null && arr == arr2)
Does this look any better? I don't think so, it means the person didn't
understand what an array actually is. Even though the second actually
has some semantic meaning (the first is a no-op), it's likely not what
the author intended.
In most cases, when they check for null, they just want to check to see
if the array is unset. The semantic meaning of this is usually that it's
empty, they don't really care if it's actually null or not. In which
case, checking for exact nullness is actually more expensive, and prone
to problems.
This comes from many languages where an array is an object type that
defaults to null, and we reinforce that misconception by allowing null
as a valid array literal.
Even people who write == null may want to check for null thinking that
it's how you check an array is empty, not realizing that it *doesn't*
check for a null pointer, *AND* it still does exactly what they need it
to do ;)
You honestly expect someone first coming to D expect to check whether an
array is empty by checking null? That's a bizarre quirk of D that I have
never seen anyhwere else. I would never expect anyone to purposefully use
== null to check for empty unless they were very familiar with D, and even
then, I'd normally expect them to ask what they really mean, which is
whether the array is empty.
Reread what I said again. They *think* they need to check if it's null
(it being the mythical Array object type that the language no doubt
lowers to, just like it does in Java or C# or Swift or...), but really,
they only need to check if it's empty. Which happens to be all they
really need.
For instance:
int[] arr;
if (cond)
{
...
arr = new int[5];
...
}
if (arr == null)
Now, you can certainly replace arr == null with arr is null, and the
code works fine -- identically, even though it's more expensive. But to
me, the arr is null is a red flag. Does the person know that they are
checking for the ACTUAL value null? You still have to read the code to
figure it out! I would say most times it's a bug waiting to happen. I
can't imagine you just see "arr is null" and move on believing the
author knew what they were doing.
It's the same reason that
if(arr)
was temporarily out of the language.
It's similar, but I consider it a different reason. While the intent of
== null may not be crystal clear, 99% of people don't care about the
pointer, they just care whether it's empty. So the default case is
usually good enough, even if you don't know the true details.
I think that that's the key point of disagreement here. I would never
consider the intent of == null to be crystal clear based solely on the code,
because it is so common outside of D to use == null to actually check for
null, and there are better ways in D to check for empty if that's what you
really mean. My immediate expectation on seeing arr == null is that the
programmer does not properly understand arrays in D. If I knew that someone
like you wrote the code, I'd probably decide that you knew what you were
doing and didn't make a mistake, but I'm not going to assume that in
general, and honestly, I would consider it bad coding practice (though we
obviously disagree on that point).
The fundamental reason why == null is generally OK is because generally
the person doesn't distinguish between nullness and non-null but empty.
Believe it or not, this is my position as well. Either works fine for
their code, and in fact, when you analyze the code, checking for
emptiness is really what they mean.
Consider that new T[0] returns a null array. What happens if it returned
a non-null array? Only code that uses "is null" would break. Code that
uses == null would work fine.
I would consider the if(arr) and arr == null cases to be exactly the same.
They both are red flags that the person in question does not understand how
arrays in D work. Yes, someone who knows what they're doing may get it
right, but I'd consider both to be code smells and I wouldn't purposefully
do either in my own code. If I found either in my own code, I would expect
that I'd just found a careless bug.
== null is way more forgiving than if(arr). That is the
Re: Can I work out error: reinterpreting cast from const(ubyte)* to const(uint)* is not supported in CTFE?
On Monday, May 21, 2018 18:13:26 Dr.No via Digitalmars-d-learn wrote:
> I'm trying to do some hashing at compile time with xxhash
> algorithm but I get this error:
>
> ..\..\..\AppData\Local\dub\packages\xxhash-master\xxhash\src\xxhash.d(39,3
> 7): Error: reinterpreting cast from const(ubyte)* to const(uint)* is not
> supported in CTFE
>
> this is line 39
>
> (https://github.com/repeatedly/xxhash-d/blob/master/src/xxhash.d#L39):
> > auto srcPtr = cast(const(uint)*)source.ptr;
>
> I'm on Windows 10 64-bit machine.
You can't do any kind of pointer manipulaton or related casts in CTFE. So,
you can't reinterpet one type as another at compile time. You would need a
different hashing algorithm for CTFE - e.g.
if(__ctfe)
{
... do something to calculate a hash
}
else
{
... calculate the hash the current way
}
- Jonathan M Davis
Re: assertNotThrown (and asserts in general)
On Monday, May 21, 2018 12:44:21 Malte via Digitalmars-d-learn wrote: > I was interested by asserts and how the compiler uses them to > optimize the code. So I looked at the compiler explorer to see > how and found it, it doesn't. > > What I tried to do is turn a std.conv.to!ulong(byte) to a simple > cast with the help of assertions. > https://godbolt.org/g/4uckWU > > If there is an assert right before an if with the same condition, > I think it should remove the compare and jump in release, but it > doesn't. I'm assuming the compilers just aren't ready yet, but > should be someday. At least that is what the documentation > promises. Walter wants to use assertions to then have the compiler make assumptions about the code and optimized based on it, but he hasn't implemented anything like that, and there are a number of arguments about why it's a very bad idea - in particular, if it allows the compiler to have undefined behavior if the assertion would have failed if it were left in. So, what is actually going to happen with that is unclear. There are folks who want additional performance benefits by allowing assertions to work as hints to the compiler, and there are folks who want them to truly just be for debugging purposes, because they don't want the compiler to then generate code that makes the function behave even more badly when the assertion would have failed but had been compiled out. Personally, my big concern is that it can't introduce undefined behavior, or it would potentially violate memory safety in @safe code, which would then mean that using assertions in @safe code could make your code effectively @system, which would defeat the whole purpose of @safe. If the compiler can optimize while not breaking those guarantees, then I'm all for it, but some folks disagree with even that, because they don't want their code to behave worse when something goes wrong. Of course, the counter-argument is that if they're that worried about it, they should leave the checks in, but there definitely isn't agreement on the matter. Walter is in favor of adding optimizations based on assertions though, so I think that there's a decent chance that we'll see something like that at some point. But language feature stuff (like implementing scope for DIP 1000) are generally much higher priority than adding extra optimizations to the compiler. So, while optimizations and performance improvements are certainly done, they're not the main focus right now. > More curious made me > "assertNotThrown!ConvOverflowException(input.to!ubyte)" where it > didn't even remove the assert code in release and produced the > most inefficient assembly. > Is that intended behaviour? In my opinion that greatly limits the > usabilty of it. assertNotThrown doesn't use any assertions. It explicitly throws an AssertError (which is what a failed assertion does when it's not compiled out). assertNotThrown would have to use a version(assert) block to version the checks to try and mirror what the assert statement does. However, assertNotThrown is specifically intended for unit tests. IIRC, assertions in unit tests are left in when compiled with -unittest (otherwise, compiling with -release and -unittest - like Phobos does for one of its passes as part of its unittest build - would not work), but I don't think that the assertions outside of unittest blocks get left in in that case, so using version(assert) on assertThrown or assertNotThrown might break them. I'm not sure. Regardless, using them for testing what assertions do is just wrong. You need to test actual assert statements if that's what you want to be testing. - Jonathan M Davis
Re: auto & class members
On Monday, May 21, 2018 11:13:16 Ali Çehreli via Digitalmars-d-learn wrote:
> On 05/20/2018 10:46 AM, Robert M. Münch wrote:
> > But I still don't understand why I can't write things explicitly but
> > have to use an alias for this.
>
> Templatized range types work well when they are used as template
> arguments themselves.
>
> When you need to keep a single type like 'b' (i.e. b is not a template),
> and when you need to set a variable like mySubStream to a dynamic
> object, the solution is to use inputObject():
>
> import std.algorithm;
> import std.range;
>
> class a {
> int[] myStream = [ 1, 2, 42, 100 ];
> }
>
>
> int myMessage = 42;
>
> class b {
> InputRange!int mySubStream;
> }
>
> void myFunc() {
> a myA = new a();
> b myB = new b();
>
> myB.mySubStream = inputRangeObject(myA.myStream.filter!(x => x ==
> myMessage));
>
> assert(myB.mySubStream.equal([myMessage]));
> }
>
> void main() {
> myFunc();
> }
>
> Now, mySubStream is a range variable that satisfies the input range
> interface and produces int elements. (Adjust accordingly.) You can use a
> more specialized range kind other than InputRange if the actual range
> supports it (e.g. ForwardRange!int, etc.):
>
>
> http://ddili.org/ders/d.en/ranges_more.html#ix_ranges_more.inputRangeObjec
> t
>
>https://dlang.org/phobos/std_range_interfaces.html#inputRangeObject
Wow. Someone actually uses those? I don't think that I've ever seen anyone
try except when they didn't understand ranges properly and thought that all
ranges derived from the interfaces in that module. I guess that they would
work in this case, but I think that the normal solution is to use typeof
(though as Robert here found, that can get a bit problematic when lambdas
get involved, whereas your solution here is pretty straightforward).
I'd be _very_ leery of using ForwardRange and the like though, since they're
going to have to allocate on every call to save, which gets expensive, and
far too often, range-based code doesn't call save correctly, meaning that
you'll often hit bugs using a forward range that's a class. Phobos is a
_lot_ better about it than it used to be, but I expect that there are still
a few such lingering bugs in there, and I'd expect the average range-based
code to screw it up. Really, the only way to get it right is to actually
test your code with reference type ranges.
If all you're using is a basic input range, then those interfaces just cost
you the one allocation and should be fine, but beyond that, I wouldn't
suggest using them if you can reasonably avoid it. And personally, I'd just
use Steven's solution of using a wrapper function so that you can ensure
that there's really only one lambda type involved, and typeof then works.
- Jonathan M Davis
Re: auto & class members
On Monday, May 21, 2018 14:55:36 Steven Schveighoffer via Digitalmars-d-
learn wrote:
> On 5/20/18 1:46 PM, Robert M. Münch wrote:
> > On 2018-05-20 17:40:39 +, Robert M. Münch said:
> >> Hi Jonathan, great! This got me a step further. So I can declare my
> >> member now. But I get an implict cast error when I try:
> >>
> >> class a {
> >> ... myStream;
> >> }
> >>
> >> class b {
> >> typeof(a.myStream.filter!(x => x == myMessage)) mySubStream;
> >> }
> >>
> >> void myFunc() {
> >> a myA = new a();
> >> b myB = new b();
> >>
> >> myB.mySubstream = myA.myStream.filter!(x => x == myMessage);
> >> }
> >>
> >> This gives (unnecessary stuff stripped):
> >>
> >> Error: cannot implicitly convert expression filter(...) of type
> >> app.myFunc.filter!(x => x == myMessage) to app.b.filter!(x => x ==
> >> myMessage)
> >
> > Answering myself: Using an alias helps.
> >
> > alias typeof(a.myStream.filter!(x => x == myMessage)) myMessageType;
>
> So the issue here is that the lambda function inside myFunc is DIFFERENT
> than the one inside b. They are both the same function, but with
> essentially different names.
>
> When you use the alias, both are using the same exact lambda.
>
> I see you are casting now as well, which looks horrible to me -- if you
> change something in your lambda now you are in for some trouble.
>
> What may make more sense (both for type sanity and for code reuse) is to
> wrap your call to filter into one place so it can be used wherever you
> need it:
>
> auto wrapStream(S)(S str) { return str.filter!(x => x == myMessage); }
>
> class b
> {
> typeof(wrapStream(a.init.myStream)()) mySubStream;
> }
>
> void myFunc() {
> a myA = new a;
> b myB = new b;
> myB.mySubstream = myA.myStream.wrapStream;
> }
That's basically what I was suggesting that he do, but I guess that I wasn't
clear enough.
- Jonathan M Davis
Re: is ==
On Monday, May 21, 2018 14:40:24 Steven Schveighoffer via Digitalmars-d- learn wrote: > On 5/21/18 2:05 PM, Jonathan M Davis wrote: > > The core problem here is that no one reading a piece of code has any way > > of knowing whether the programmer knew what they were doing or not when > > using == null with an array, and the vast majority of newbies are not > > going to have understood the semantics properly. If I know that someone > > like you or Andrei wrote the code, then the odds are good that what the > > code does is exactly what you intended. But for the average D > > programmer? I don't think that it makes any sense to assume that, > > especially since anyone coming from another language is going to assume > > that == null is checking for null, when it's not. > > For me, the code smell is using arr is null (is it really necessary to > check for a null pointer here?), for which I always have to look at more > context to see if it's *really* right. Really? I would never expect anyone to use is unless they really cared about whether array was null. I'd be concerned about whether the code in general was right, because treating null as special gets tricky, but that particular line wouldn't concern me. > Even people who write == null may want to check for null thinking that > it's how you check an array is empty, not realizing that it *doesn't* > check for a null pointer, *AND* it still does exactly what they need it > to do ;) You honestly expect someone first coming to D expect to check whether an array is empty by checking null? That's a bizarre quirk of D that I have never seen anyhwere else. I would never expect anyone to purposefully use == null to check for empty unless they were very familiar with D, and even then, I'd normally expect them to ask what they really mean, which is whether the array is empty. > > It's the same reason that > > > > if(arr) > > > > was temporarily out of the language. > > It's similar, but I consider it a different reason. While the intent of > == null may not be crystal clear, 99% of people don't care about the > pointer, they just care whether it's empty. So the default case is > usually good enough, even if you don't know the true details. I think that that's the key point of disagreement here. I would never consider the intent of == null to be crystal clear based solely on the code, because it is so common outside of D to use == null to actually check for null, and there are better ways in D to check for empty if that's what you really mean. My immediate expectation on seeing arr == null is that the programmer does not properly understand arrays in D. If I knew that someone like you wrote the code, I'd probably decide that you knew what you were doing and didn't make a mistake, but I'm not going to assume that in general, and honestly, I would consider it bad coding practice (though we obviously disagree on that point). I would consider the if(arr) and arr == null cases to be exactly the same. They both are red flags that the person in question does not understand how arrays in D work. Yes, someone who knows what they're doing may get it right, but I'd consider both to be code smells and I wouldn't purposefully do either in my own code. If I found either in my own code, I would expect that I'd just found a careless bug. > > At this point, I'm honestly inclined to think that we never should have > > allowed null for arrays. We should have taken the abstraction a bit > > further and disallowed using null to represent dynamic arrays. It would > > then presumably still work to do arr.ptr is null, but arr is null > > wouldn't work, because null wouldn't be an array, and arr == null > > definitely wouldn't work. Then we could just use [] for empty arrays > > everywhere, and there would be no confusion, leaving null for actual > > pointers. And it would almost certinly kill off all of the cases where > > null was treated as special for dynamic arrays except maybe for when > > dealing with C code, but in that case, they'd have to use ptr directly. > > However, at this point, I expect that that's all water under the > > bridge, and we're stuck with it. > > If we never had null be the default value for an array, and used [] > instead, I would be actually OK with that. I also feel one of the > confusing things for people coming to the language is that arrays are > NOT exactly reference types, even though null can be used as a value for > assignment or comparison. > > But it still wouldn't change what most people write or mean, they just > would write == [] instead of == null. I don't see how this would solve > any of your concerns. It would solve the concern, because no one is going to write arr == [] to check for null. They'de write it just like they'd write arr == "". They're clearly checking for empty, not null. The whole problem here is that pretty much everywhere other than D arrays, null and empty are two separate things, and pretty much anyone coming from
Re: auto & class members
On 5/20/18 1:46 PM, Robert M. Münch wrote:
On 2018-05-20 17:40:39 +, Robert M. Münch said:
Hi Jonathan, great! This got me a step further. So I can declare my
member now. But I get an implict cast error when I try:
class a {
... myStream;
}
class b {
typeof(a.myStream.filter!(x => x == myMessage)) mySubStream;
}
void myFunc() {
a myA = new a();
b myB = new b();
myB.mySubstream = myA.myStream.filter!(x => x == myMessage);
}
This gives (unnecessary stuff stripped):
Error: cannot implicitly convert expression filter(...) of type
app.myFunc.filter!(x => x == myMessage) to app.b.filter!(x => x ==
myMessage)
Answering myself: Using an alias helps.
alias typeof(a.myStream.filter!(x => x == myMessage)) myMessageType;
So the issue here is that the lambda function inside myFunc is DIFFERENT
than the one inside b. They are both the same function, but with
essentially different names.
When you use the alias, both are using the same exact lambda.
I see you are casting now as well, which looks horrible to me -- if you
change something in your lambda now you are in for some trouble.
What may make more sense (both for type sanity and for code reuse) is to
wrap your call to filter into one place so it can be used wherever you
need it:
auto wrapStream(S)(S str) { return str.filter!(x => x == myMessage); }
class b
{
typeof(wrapStream(a.init.myStream)()) mySubStream;
}
void myFunc() {
a myA = new a;
b myB = new b;
myB.mySubstream = myA.myStream.wrapStream;
}
-Steve
Re: UFCS syntax I never saw before.
On Monday, May 21, 2018 14:00:55 ANtlord via Digitalmars-d-learn wrote: > On Monday, 21 May 2018 at 11:38:12 UTC, SrMordred wrote: > > After all this time I saw this: > > > > writeln = iota = 5; > > > > what?? > > > > I never saw that before! > > > > This is interesting, there is something useful that i can do > > with this kind of call? > > What the hell is this? I don't figure out why are there so many > syntax features? It doesn't make the language more easier. Is it > any reason to support 2 additional cases of function calling? We > have no literal for a tuple but we call a function via 3 cases. > I'm frustrated. This particular example is a result of how property functions were originally introduced into the language, and UCFS just compounded the problem. It's also why () is optional on function calls. foo = bar; is treating foo like a setter property, and auto result = foo; is treating foo like a getter property. Heck, in foo = bar; both foo and bar could be function calls. As long as it's used intelligently, this is great, because then you can treat functions as properties. But it gets really bad when someone does something dumb like writeln = "hello"; Because this can get really dumb, @property was introduced into the language with the idea that it would be used to mark functions which were supposed to be used as properties. Then dumb stuff like writeln = "hello"; could become illegal, whereas stuff functions that were actually intended to be used as properties could continue to be used as properties. So, the clean uses would be left, and the bad uses would be illegal. Unfortunately, because moving to @property meant breaking code (even code that was using properties legitimately, since it was introduced before @property was a thing), that meant that we had to introduce @property as doing nothing and then switch to enforcing it later. Somewhere along the line, the -property flag was introduced to start enforcing it, but it only partially enforced it, and of course code continued to be written without using it. So, actually moving to enforcing everything with @property was slow. And then UFCS happened, and that pretty much killed the whole thing. The problem was twofold: 1. A number of folks just got in the habit of calling functions without parens and liked it. They would generally agree that writeln = "foo"; was dumb, but they liked being able to just not use parens when doing stuff like myObj = foo; or myFunc(foo); So, telling them that they couldn't do that anymore didn't go over well. 2. Once, UFCS came into the game, instead of having code like auto result = map!(a => a / 2)(range); you got auto result = range.map!(a => / 2)(); and you all of a sudden had a bunch of templated functions with empty parens when called, and since you already had a one set of parens for the template argument, folks thought that that was ugly. So, they started writing auto result = range.map!(a => / 2); So, requiring that they then use parens for that as they would have to if parens were required when calling all non-@property functions as had been the plan was _not_ going to go over well. It basically became completely unacceptable to require @property for property functions, and so plans to properly implement @property were dropped. The result is that @property does almost nothing (it affects what typeof does, and it affects __traits - and thus std.traits.functionAttributes - and you can't overload an @property function and a normal function, but that's pretty much it). Lots of us use it all the time to indicate what we intend to be used as a property function, but it's just documentation. Now, even if you think that calling functions with no parens is great, that still leaves us with two problems: 1. writeln = "foo"; is still legal. Maybe we could make it so that @property is required with free functions used as setters that aren't used with UFCS, but figuring out a set of rules that doesn't require putting @property on all setter functions while still disallowing the really dumb stuff isn't easy, and it's questionable that requiring @property on setters at this point is going to go over well. It's very annoying that stuff like writeln = "foo"; is legal, and it's dumb, but it hasn't mattered much in practice. So, causing a bunch of code breakage in order to disallow it is unlikely to go over well. It would also then make getters and setters inconsistent in that setters would require @property and getters wouldn't. How much that matters is debatable, but it does make such a change less palatable. 2. The other issue that @property was supposed to fix was property functions that return callables - e.g. foo could return a delegate, but the compiler has to assume that the parens on foo() are calling foo, not the delegate that it returns. So, even though you want to treat foo as a property that is a delegate, you're forced to treat it as a function and use double parens - e.g. foo()().
Re: is ==
On 5/21/18 2:05 PM, Jonathan M Davis wrote: The core problem here is that no one reading a piece of code has any way of knowing whether the programmer knew what they were doing or not when using == null with an array, and the vast majority of newbies are not going to have understood the semantics properly. If I know that someone like you or Andrei wrote the code, then the odds are good that what the code does is exactly what you intended. But for the average D programmer? I don't think that it makes any sense to assume that, especially since anyone coming from another language is going to assume that == null is checking for null, when it's not. For me, the code smell is using arr is null (is it really necessary to check for a null pointer here?), for which I always have to look at more context to see if it's *really* right. Even people who write == null may want to check for null thinking that it's how you check an array is empty, not realizing that it *doesn't* check for a null pointer, *AND* it still does exactly what they need it to do ;) It's the same reason that if(arr) was temporarily out of the language. It's similar, but I consider it a different reason. While the intent of == null may not be crystal clear, 99% of people don't care about the pointer, they just care whether it's empty. So the default case is usually good enough, even if you don't know the true details. Whereas, if(arr) is checking that the pointer is null as well as the length is 0. Most people aren't expecting that, and those who are are like Andrei and Vladimir -- they know the quirks of the language here. For the longest time I thought it was just checking the pointer! I think aside from the clout of the ones who wanted it, the biggest reason that change was reverted was that it became really difficult to use an array inside a conditional. One-liners had to be extracted out, temporary variables defined. At this point, I'm honestly inclined to think that we never should have allowed null for arrays. We should have taken the abstraction a bit further and disallowed using null to represent dynamic arrays. It would then presumably still work to do arr.ptr is null, but arr is null wouldn't work, because null wouldn't be an array, and arr == null definitely wouldn't work. Then we could just use [] for empty arrays everywhere, and there would be no confusion, leaving null for actual pointers. And it would almost certinly kill off all of the cases where null was treated as special for dynamic arrays except maybe for when dealing with C code, but in that case, they'd have to use ptr directly. However, at this point, I expect that that's all water under the bridge, and we're stuck with it. If we never had null be the default value for an array, and used [] instead, I would be actually OK with that. I also feel one of the confusing things for people coming to the language is that arrays are NOT exactly reference types, even though null can be used as a value for assignment or comparison. But it still wouldn't change what most people write or mean, they just would write == [] instead of == null. I don't see how this would solve any of your concerns. -Steve
Re: Real Int24
On Monday, 21 May 2018 at 15:41:21 UTC, Simen Kjærås wrote:
On Saturday, 19 May 2018 at 18:44:42 UTC, IntegratedDimensions
wrote:
On Saturday, 19 May 2018 at 18:19:35 UTC, IntegratedDimensions
wrote:
Is there any way to create an int24 type that behaves just
like any other built in type without having to reimplement
everything?
In fact, what I'd like to do is create an arbitrary type:
struct atype(T)
{
}
where atype(T) is just a "view" in to N_T bits interpreted as
T, an enum.
If T is bit, then the N = 1 and the interpretation is 1 bit.
If T is byte, then the N = 8 and the interpretation is 7 bits
followed by 1 signed bit.
If T is int24, then the N = 24 and the interpretation is 23
bits followed by 1 signed bit.
The idea is the storage of atype is exactly N bits. If this is
not possible due to boundary issues then N can always be a
multiple of 8(which is for my use cause is the smallest).
D does not support types that take up less than one byte of
space. It's possible to make types that represent less than one
byte - bool may be considered such an example - but they still
take up at least 1 byte.
If you create a custom range type, you could pack more than one
element in each byte, see std.bitmanip.BitArray[0] for an
example.
The main thing is that I would like to be able to use atype as
if it were a built in type.
If N = 24, 3 bytes, I want to be able to create arrays of
atype!int24[] which work just as if they were arrays of bytes
without any exception or special cases.
atype!byte would be equivalent to byte and reduce to the
compiler internals. I'm not looking to create a "view" of an
array. I want a standalone type that can behave as all the
desired types needed, which is most of the numerical types of
D and some of the ones it neglected like 24-bit ints, 48-bit
ints, etc. Ideally, any type could be used and the "most
optimal" code is generated while being able to use the types
using the standard model.
We already have std.numeric.CustomFloat[1]. As the name
implies, it only works for floats.
I hacked together something somewhat equivalent for ints:
https://gist.github.com/Biotronic/f6668d8ac95b70302015fee93ae9c8c1
Usage:
// Two's-complement, native-endian, 24-bit int type:
CustomInt!24 a;
// Unsigned, native-endian, 15-bit:
CustomInt!(15, Representation.Unsigned) b;
// Offset (-2..5) 3-bit int:
CustomInt!(3, Representation.OffsetBinary, 2) c;
// You get the idea:
CustomInt!(64, Representation.SignedMagnitude, 0,
Endianness.BigEndian) d;
Not sure this is what you're looking for, but it's at the very
least inspired by your post. :)
If what you want is a type that can represent something a
packed array of 13-bit ints, the above is not what you're
looking for - you're going to need a custom range type.
--
Simen
[0]: https://dlang.org/phobos/std_bitmanip#BitArray
[1]: https://dlang.org/phobos/std_numeric.html#.CustomFloat
Cool. I'll try it as a drop in replacement and if it works then
it works! ;) Thanks.
Just to be clear and to make sure this works the way it seems:
All types multiple of a byte are reduced to either internal
representation(byte, ubyte, short, ushort, int, uint, long,
ulong) directly(becomes an alias) or the most efficient structure
for the representation: unsigned 24-bit = 3 bytes and is
effectively ubyte[3], usigned 128-bit is ubyte[16], etc?
Non multiples are extended up one byte, so 7-bits is representing
using an byte, etc.
This seems to be the case from the code.
Now, what I didn't see was anything to work with non byte aligned
arrays of CustomInt. Would it be possible to add? I know you say
that we should use bitmanip but code could be extended to support
it relatively easily by treating an array of bits as an array of
CustomInts and the indexer can compute the appropriate offset
using the bit size.
Maybe that will require CustomIntsArray?
The idea is, say one has 7-bit ASCII represented in a ubyte[]
then they can map that to a CustomInt!7[] which will be use
CustomInt!7(but 7 bits, not 8) as representation. But, of course
CustomInt!7[3] would be 8 bits. But basically it retrieves the
correct value and stores it by doing the standard masking.
BTW, it looks like you could extend your type to deal with floats
and doubles which would make this type very robust in dealing
with arbitrary primitive types.
The idea is that any matching language types are aliased to and
those that don't are handled appropriately.
Can I work out error: reinterpreting cast from const(ubyte)* to const(uint)* is not supported in CTFE?
I'm trying to do some hashing at compile time with xxhash algorithm but I get this error: ..\..\..\AppData\Local\dub\packages\xxhash-master\xxhash\src\xxhash.d(39,37): Error: reinterpreting cast from const(ubyte)* to const(uint)* is not supported in CTFE this is line 39 (https://github.com/repeatedly/xxhash-d/blob/master/src/xxhash.d#L39): auto srcPtr = cast(const(uint)*)source.ptr; I'm on Windows 10 64-bit machine.
Re: auto & class members
On 05/20/2018 10:46 AM, Robert M. Münch wrote:
> But I still don't understand why I can't write things explicitly but
> have to use an alias for this.
Templatized range types work well when they are used as template
arguments themselves.
When you need to keep a single type like 'b' (i.e. b is not a template),
and when you need to set a variable like mySubStream to a dynamic
object, the solution is to use inputObject():
import std.algorithm;
import std.range;
class a {
int[] myStream = [ 1, 2, 42, 100 ];
}
int myMessage = 42;
class b {
InputRange!int mySubStream;
}
void myFunc() {
a myA = new a();
b myB = new b();
myB.mySubStream = inputRangeObject(myA.myStream.filter!(x => x ==
myMessage));
assert(myB.mySubStream.equal([myMessage]));
}
void main() {
myFunc();
}
Now, mySubStream is a range variable that satisfies the input range
interface and produces int elements. (Adjust accordingly.) You can use a
more specialized range kind other than InputRange if the actual range
supports it (e.g. ForwardRange!int, etc.):
http://ddili.org/ders/d.en/ranges_more.html#ix_ranges_more.inputRangeObject
https://dlang.org/phobos/std_range_interfaces.html#inputRangeObject
Ali
Re: is ==
On Monday, May 21, 2018 10:01:15 Steven Schveighoffer via Digitalmars-d- learn wrote: > On 5/18/18 9:48 PM, Jonathan M Davis wrote: > > Of > > course, the most notable case where using == with null is a terrible > > idea is dynamic arrays, and that's the case where the compiler > > _doesn't_ complain. > I use arr == null all the time. I'm perfectly fine with that, and > understand what it means. > > > Using == with null and arrays is always unclear about the programmer's > > intent and almost certainly wasn't what the programmer intended. > > I beg to differ. > > > If the > > programmer cares about null, they should use is. If they care about > > lengnth, then that's what they should check. Checking null with == is > > just a huge code smell. > > IMO, doing anything based on the pointer of an array being null is a > huge code smell. In which case, == null is perfectly acceptable. I'm > comparing my array to an empty array. What is confusing about that? > > I actually hate using the pointer in any aspect -- an array is > semantically equivalent to its elements, it's not important where it's > allocated. The only place D forces me to care about the pointer is when > I'm dealing with ranges. The core problem here is that no one reading a piece of code has any way of knowing whether the programmer knew what they were doing or not when using == null with an array, and the vast majority of newbies are not going to have understood the semantics properly. If I know that someone like you or Andrei wrote the code, then the odds are good that what the code does is exactly what you intended. But for the average D programmer? I don't think that it makes any sense to assume that, especially since anyone coming from another language is going to assume that == null is checking for null, when it's not. It's the same reason that if(arr) was temporarily out of the language. The odds are very high that the programmer using it is using it wrong. Andrei and Vladimir were using it correctly in their code, so they didn't like the fact that it had then become illegal, but while knew what they were doing and were using it correctly, plenty of other folks have been inserting bugs whenever they do that, and if I see if(arr) or assert(arr) in code, I'm going to consider it to be code smell just as much as I consider arr == null to be code smell. And yes, trying to treat the ptr as being null as special with a dynamic array is risky, and most code shouldn't be doing it, but you're almost forced to in some cases when interacting with C code, and clearly there are folks that do (e.g. Andrei and Vladimir). But even if we could unequivocably say that no one should be doing it, you still have no way of knowing whether someone is attempting it or not when they do arr == null, and since caring whether an array is null or not is a very typical thing to do in other languages where there is a very clear distinction between a null array and an empty one, plenty of folks come to D expecting to be able to do the same. And they're going to write arr == null or arr != null, and any time I see code like that, I'm going to have sit down and figure out whether they really meant arr is null, or whether they meant arr.length == 0, whereas if they had just written arr is null or arr.length == 0 (or arr.empty), their intent would have been perfectly clear. As such, I would strongly advise D programmers to use arr.empty or arr.length == 0 instead of arr == null, even if they know what they're doing, just like I would advise them to not treat null as special for arrays unless they really need to. At this point, I'm honestly inclined to think that we never should have allowed null for arrays. We should have taken the abstraction a bit further and disallowed using null to represent dynamic arrays. It would then presumably still work to do arr.ptr is null, but arr is null wouldn't work, because null wouldn't be an array, and arr == null definitely wouldn't work. Then we could just use [] for empty arrays everywhere, and there would be no confusion, leaving null for actual pointers. And it would almost certinly kill off all of the cases where null was treated as special for dynamic arrays except maybe for when dealing with C code, but in that case, they'd have to use ptr directly. However, at this point, I expect that that's all water under the bridge, and we're stuck with it. - Jonathan M Davis
Re: scope guards & debugger breakpoints?
On 2018-05-21 17:24:12 +, Steven Schveighoffer said:
I'm not 100% sure but I expect:
scope(failure)
someCode();
putting a breakpoint on someCode should work.
When calling a function an then setting the breakpoint there, like in
someCode() yes, that should work.
I used code like this:
scope(failure) {
... several lines of code
}
And for this it seems (maybe I need to check in more detail) that in
MSVC, I can set a breakpoint, but the breakpoint is not triggered when
an exception is thrown.
--
Robert M. Münch
http://www.saphirion.com
smarter | better | faster
VisualD / fatal error C1905: Front-End and Back-End are not compatible (have to use the same processor)
A project I can compile via the command line and dub, gives an error in VisualD. I created the VisualD configuration through dub: fatal error C1905: Front-End und Back-End sind nicht kompatibel (müssen den gleichenProzessor verwenden). This translates to: "Front-End and Back-End are not compatible (have to use the same processor)" Well, I don't have a clue what this should mena, nor how this could happen. It sounds a bit like if the compiler & linker are not useing the same architecture (I want to use x64) but I didn't find any options to check/change this. Any ideas? -- Robert M. Münch http://www.saphirion.com smarter | better | faster
Re: Splitting up large dirty file
On Monday, May 21, 2018 15:00:09 Dennis via Digitalmars-d-learn wrote:
> On Thursday, 17 May 2018 at 21:10:35 UTC, Dennis wrote:
> > It's unfortunate that Phobos tells you 'there's problems with
> > the encoding' without providing any means to fix it or even
> > diagnose it.
>
> I have to take that back since I found out about std.encoding
> which has functions like `sanitize`, but also `transcode`. (My
> file turned out to actually be encoded with ANSI / Windows-1252,
> not UTF-8)
> Documentation is scarce however, and it requires strings instead
> of forward ranges.
>
> @Jon Degenhardt
>
> > Instead of:
> > auto outputFile = new File("output.txt");
> >
> > try:
> > auto outputFile = File("output.txt", "w");
>
> Wow I really butchered that code. So it is the `drop(4)` that
> triggers the UTFException?
drop is range-based, so if you give it a string, it's going to decode
because of the whole auto-decoding mess with std.range.primitives.front and
popFront. If you can't have auto-decoding, you either have to be dealing
with functions that you know avoid it, or you need to do use something like
std.string.representation or std.utf.byCodeUnit to get around the
auto-decoding. If you're dealing with invalid Unicode, you basically have to
either convert it all up front or do something like treat it like binary
data, or Phobos is going to try to decode it as Unicode and give you a
UTFExceptions.
> I find Exceptions in range code hard to interpret.
Well, if you just look at the stack trace, it should tell you. I don't see
why ranges would be any worse than any other code except for maybe the fact
that it's typical to chain a lot of calls, and you frequently end up with
wrapper types in the stack trace that you're not necessarily familiar with.
The big problem here really is that all you're really being told is that
your string has invalid Unicode in it somewhere and the chain of function
calls that resulted in std.utf.decode being called on your invalid Unicode.
But even if you weren't dealing with ranges, if you passed invalid Unicode
to something completely string-based which did decoding, you'd run into
pretty much the same problem. The data is being used outside of its original
context where you could easily figure out what it relates to, so it's going
to be a problem by its very nature. The only real solution there is to be
controlling the decoding yourself, and even then, it's easy to be in a
position where it's hard to figure out where in the data the bad data is
unless you've done something like keep track of exactly what index your at,
which really doesn't work well once you're dealing with slicing data.
> @Kagamin
>
> > Do it old school?
>
> I want to be convinved that Range programming works like a charm,
> but the procedural approaches remain more flexible (and faster
> too) it seems. Thanks for the example.
The whole auto-decoding mess makes things worse than they should be, but if
you find procedural examples more flexible, then I would guess that that
would be simply a matter of getting more experience with ranges. Ranges are
far more composable in terms of how they're used, which tends to inherently
make them more flexible. However, it does result in code that's a mixture of
functional and procedural programming, which can be quite a shift for some
folks. So, there's no question that it takes some getting used to, but D
does allow for the more classic approaches, and ranges are not always the
best approach.
As for performance, that depends on the code and the compiler. It wouldn't
surprise me if dmd didn't optimize out the range stuff as much as it really
should, but it's my understanding that ldc typically manages to generate
code where the range abstraction didn't cost you anything. If there's an
issue, I think that it's frequently an algorithmic one or the fact that some
range-processing has a tendency to process the same data multiple times,
because that's the easiest, most abstract way to go about it and works in
general but isn't always the best solution.
For instance, because of how the range API works, when using splitter, if
you iterate through the entire range, you pretty much have to iterate
through it twice, because it does look-ahead to find the delimiter and then
returns you a slice up to that point, after which, you process that chunk of
the data to do whatever it is you want to do with each split piece. At a
conceptual level, what you're doing with your code with splitter is then
really clean and easy to write, and often, it should be plenty efficient,
but it does require going over the data twice, whereas if you looped over
the data yourself, looking for each delimiter, you'd only need to iterate
over it once. So, in cases like that, I'd fully expect the abstraction to
cost you, though whether it costs enough to matter depends on what you're
doing.
As is the case when dealing with most abstractions, I think that it's mostly
a matter of using it where it makes sense
Re: scope guards & debugger breakpoints?
On 5/21/18 1:00 PM, Robert M. Münch wrote: If I use scope(failure) with code that should be run if an exception is thrown, how can I set a breakpoint for this code in the debugger? I'm not 100% sure but I expect: scope(failure) someCode(); putting a breakpoint on someCode should work. -Steve
scope guards & debugger breakpoints?
If I use scope(failure) with code that should be run if an exception is thrown, how can I set a breakpoint for this code in the debugger? -- Robert M. Münch http://www.saphirion.com smarter | better | faster
Re: Temporary file creation for unittests
On Mon, 2018-05-21 at 15:16 +, Atila Neves via Digitalmars-d-learn wrote:
> On Friday, 18 May 2018 at 15:16:52 UTC, Russel Winder wrote:
> > Hi,
> >
> > What's the current official position on how to create temporary
> > files for use during a unittest. I found
>
> Not official, but...
>
> import unit_threaded;
>
> with(const Sandbox()) {
> writeFile("myfile.txt", "contents");
> shouldExist("myfile.txt");
> shouldEqualContent("myfile.txt", "contents");
> fileShouldContain("myfile.txt", "cont");
> }
>
> Atila
OK, we like this. A lot.
Given I use Unit-Threaded, why did I not know this. Ah, OK, RTFM. :-)
Did I mention how much I like this RAII approach?
--
Russel.
==
Dr Russel Winder t: +44 20 7585 2200
41 Buckmaster Roadm: +44 7770 465 077
London SW11 1EN, UK w: www.russel.org.uk
signature.asc
Description: This is a digitally signed message part
Re: UFCS syntax I never saw before.
"%s %s".writefln = ("foo".tuple = "bar").expand;
lol
How do I see the flags passed from dub to the compiler?
where's this stored?
Re: How do I see the flags passed from dub to the compiler?
On 22/05/2018 4:21 AM, Dr.No wrote: where's this stored? -v should do the trick
Re: Real Int24
On Saturday, 19 May 2018 at 18:44:42 UTC, IntegratedDimensions
wrote:
On Saturday, 19 May 2018 at 18:19:35 UTC, IntegratedDimensions
wrote:
Is there any way to create an int24 type that behaves just
like any other built in type without having to reimplement
everything?
In fact, what I'd like to do is create an arbitrary type:
struct atype(T)
{
}
where atype(T) is just a "view" in to N_T bits interpreted as
T, an enum.
If T is bit, then the N = 1 and the interpretation is 1 bit.
If T is byte, then the N = 8 and the interpretation is 7 bits
followed by 1 signed bit.
If T is int24, then the N = 24 and the interpretation is 23
bits followed by 1 signed bit.
The idea is the storage of atype is exactly N bits. If this is
not possible due to boundary issues then N can always be a
multiple of 8(which is for my use cause is the smallest).
D does not support types that take up less than one byte of
space. It's possible to make types that represent less than one
byte - bool may be considered such an example - but they still
take up at least 1 byte.
If you create a custom range type, you could pack more than one
element in each byte, see std.bitmanip.BitArray[0] for an example.
The main thing is that I would like to be able to use atype as
if it were a built in type.
If N = 24, 3 bytes, I want to be able to create arrays of
atype!int24[] which work just as if they were arrays of bytes
without any exception or special cases.
atype!byte would be equivalent to byte and reduce to the
compiler internals. I'm not looking to create a "view" of an
array. I want a standalone type that can behave as all the
desired types needed, which is most of the numerical types of D
and some of the ones it neglected like 24-bit ints, 48-bit
ints, etc. Ideally, any type could be used and the "most
optimal" code is generated while being able to use the types
using the standard model.
We already have std.numeric.CustomFloat[1]. As the name implies,
it only works for floats.
I hacked together something somewhat equivalent for ints:
https://gist.github.com/Biotronic/f6668d8ac95b70302015fee93ae9c8c1
Usage:
// Two's-complement, native-endian, 24-bit int type:
CustomInt!24 a;
// Unsigned, native-endian, 15-bit:
CustomInt!(15, Representation.Unsigned) b;
// Offset (-2..5) 3-bit int:
CustomInt!(3, Representation.OffsetBinary, 2) c;
// You get the idea:
CustomInt!(64, Representation.SignedMagnitude, 0,
Endianness.BigEndian) d;
Not sure this is what you're looking for, but it's at the very
least inspired by your post. :)
If what you want is a type that can represent something a packed
array of 13-bit ints, the above is not what you're looking for -
you're going to need a custom range type.
--
Simen
[0]: https://dlang.org/phobos/std_bitmanip#BitArray
[1]: https://dlang.org/phobos/std_numeric.html#.CustomFloat
Re: Temporary file creation for unittests
On Monday, 21 May 2018 at 15:16:11 UTC, Atila Neves wrote:
On Friday, 18 May 2018 at 15:16:52 UTC, Russel Winder wrote:
Hi,
What's the current official position on how to create
temporary files for use during a unittest. I found
Not official, but...
import unit_threaded;
with(const Sandbox()) {
writeFile("myfile.txt", "contents");
shouldExist("myfile.txt");
shouldEqualContent("myfile.txt", "contents");
fileShouldContain("myfile.txt", "cont");
}
Atila
I've never seen "should" being in used in function names before...
Re: Mysql query result access by field name
On 5/21/18 10:59 AM, kdevel wrote:
On Monday, 21 May 2018 at 14:17:23 UTC, Steven Schveighoffer wrote:
Data f;
allrows[0].toStruct (f);
I haven't checked this.
This only works if your struct has exactly the same layout as the fields.
So if, for instance, your rows are selected "title", "name",
"surname", but your data type orders them name, surname, title, you
won't be happy with the result.
Haven't seen this. Then there is no more field-safety than in the OP's
"assembler" code.
In the other post you wrote
| 1. Use ResultRange instead of the Row interface. This provides
| a couple of ways to use column names, .asAA to get all the data
| in a nice AA format (they are still variants),
The AA format can than be used to fill the struct automatically
(detecting missing and excess fields) like in this code:
T toStructX(T) (string[string] a)
{
T t;
bool[string] bookkeep;
foreach (i, m; t.tupleof) {
string key = T.tupleof[i].stringof;
if (key !in a) {
stderr.writefln ("missing key <%s>", key);
continue;
}
t.tupleof[i] = a[key].to!(typeof (m));
bookkeep[key] = true;
}
foreach (x, y; a)
if (x !in bookkeep)
stderr.writefln ("excess key-value pair <%s>:<%s>", x, y);
return t;
}
Yes, this is very similar to what I do in my serialization library,
except I don't use the AA, I keep a map of indexes based on the index of
the field in the tuple to avoid allocation associated with an AA.
Note that your incoming AA is going to be a Variant[string].
-Steve
Re: Temporary file creation for unittests
On Friday, 18 May 2018 at 15:16:52 UTC, Russel Winder wrote:
Hi,
What's the current official position on how to create temporary
files for use during a unittest. I found
Not official, but...
import unit_threaded;
with(const Sandbox()) {
writeFile("myfile.txt", "contents");
shouldExist("myfile.txt");
shouldEqualContent("myfile.txt", "contents");
fileShouldContain("myfile.txt", "cont");
}
Atila
Re: Splitting up large dirty file
On Thursday, 17 May 2018 at 21:10:35 UTC, Dennis wrote:
It's unfortunate that Phobos tells you 'there's problems with
the encoding' without providing any means to fix it or even
diagnose it.
I have to take that back since I found out about std.encoding
which has functions like `sanitize`, but also `transcode`. (My
file turned out to actually be encoded with ANSI / Windows-1252,
not UTF-8)
Documentation is scarce however, and it requires strings instead
of forward ranges.
@Jon Degenhardt
Instead of:
auto outputFile = new File("output.txt");
try:
auto outputFile = File("output.txt", "w");
Wow I really butchered that code. So it is the `drop(4)` that
triggers the UTFException? I find Exceptions in range code hard
to interpret.
@Kagamin
Do it old school?
I want to be convinved that Range programming works like a charm,
but the procedural approaches remain more flexible (and faster
too) it seems. Thanks for the example.
Re: Mysql query result access by field name
On Monday, 21 May 2018 at 14:17:23 UTC, Steven Schveighoffer
wrote:
Data f;
allrows[0].toStruct (f);
I haven't checked this.
This only works if your struct has exactly the same layout as
the fields.
So if, for instance, your rows are selected "title", "name",
"surname", but your data type orders them name, surname, title,
you won't be happy with the result.
Haven't seen this. Then there is no more field-safety than in the
OP's "assembler" code.
In the other post you wrote
| 1. Use ResultRange instead of the Row interface. This provides
| a couple of ways to use column names, .asAA to get all the data
| in a nice AA format (they are still variants),
The AA format can than be used to fill the struct automatically
(detecting missing and excess fields) like in this code:
T toStructX(T) (string[string] a)
{
T t;
bool[string] bookkeep;
foreach (i, m; t.tupleof) {
string key = T.tupleof[i].stringof;
if (key !in a) {
stderr.writefln ("missing key <%s>", key);
continue;
}
t.tupleof[i] = a[key].to!(typeof (m));
bookkeep[key] = true;
}
foreach (x, y; a)
if (x !in bookkeep)
stderr.writefln ("excess key-value pair <%s>:<%s>",
x, y);
return t;
}
Re: UFCS syntax I never saw before.
On Monday, 21 May 2018 at 11:38:12 UTC, SrMordred wrote:
what??
Here's another weird example:
```
void funWithUfcsAndPropertySyntax() {
import std.typecons : tuple;
"%s %s".writefln = ("foo".tuple = "bar").expand;
}
```
source:
https://github.com/Hackerpilot/Idiotmatic-D/blob/master/idiotmatic.d#L78
Re: UFCS syntax I never saw before.
On Monday, 21 May 2018 at 11:38:12 UTC, SrMordred wrote: After all this time I saw this: writeln = iota = 5; what?? I never saw that before! This is interesting, there is something useful that i can do with this kind of call? That's pretty cool, but at the same time this should be wiped off the face of the Earth.
Re: Mysql query result access by field name
On 5/21/18 9:39 AM, kdevel wrote: On Sunday, 20 May 2018 at 16:08:03 UTC, ipkwena wrote: How does one access the columns fields in a Mysql query results by the column name. [...] Data f; f.name = to!string(allrows[0][0]); f.surname = to!string(allrows[0][1]); f.title = to!string(allrows[0][2]); I am using the mysql-native package or DB connectivity. According to the source code https://github.com/mysql-d/mysql-native/blob/master/source/mysql/result.d it should be possible to write the rowdata into the struct: Data f; allrows[0].toStruct (f); I haven't checked this. This only works if your struct has exactly the same layout as the fields. So if, for instance, your rows are selected "title", "name", "surname", but your data type orders them name, surname, title, you won't be happy with the result. -Steve
Re: UFCS syntax I never saw before.
On 5/21/18 8:15 AM, SrMordred wrote:
Right, so this should´n be working I think.
struct SomeStruct
{
void foo(int);
}
SomeStruct s;
s.foo = 10;
I thought that only with @property this will work.
That was the plan, but it got derailed.
Whoever wrote that original line of code, they need a stern talking-to.
-Steve
Re: Mysql query result access by field name
On 5/20/18 12:08 PM, ipkwena wrote: I have started learning D and I am enjoying it so far. How does one access the columns fields in a Mysql query results by the column name. Currently I have to use the method as shown in a couple of example by indexing array values (f being a struct variable): Data f; f.name = to!string(allrows[0][0]); f.surname = to!string(allrows[0][1]); f.title = to!string(allrows[0][2]); I am using the mysql-native package or DB connectivity. This is one of the weak spots of mysql-native -- the Row object has no knowledge of the column names, so you have to "know" the order of the columns you got from the server. So what you can do is: 1. Use ResultRange instead of the Row interface. This provides a couple of ways to use column names, .asAA to get all the data in a nice AA format (they are still variants), .colNames to get the list of column names ordered by the row fields, or .colNameIndicies which gives you an AA of names to indices. Note that the AA generating versions will allocate a lot of throw-away data. 2. Write a complicated serialization library like I did :) In this case, I'm turning my ResultRange from a range of Rows to a range of the data type I want, all serialized by column name instead of index. Unfortunately, this is not open source so I can't share it. At some point, I want to fix this part of mysql-native. I'm a bit annoyed that we have to do Variants instead of writing directly to the data type we are going to use anyway. -Steve
Re: is ==
On 5/18/18 9:48 PM, Jonathan M Davis wrote: On Saturday, May 19, 2018 01:27:59 Neia Neutuladh via Digitalmars-d-learn wrote: On Friday, 18 May 2018 at 23:53:12 UTC, IntegratedDimensions wrote: Why does D complain when using == to compare with null? Is there really any technical reason? if one just defines == null to is null then there should be no problem. It seems like a pedantic move by who ever implemented it and I'm hoping there is actually a good technical reason for it. tldr: this error is outdated. In the days of yore, "obj == null" would call "obj.opEquals(null)". Attempting to call a virtual method on a null object is a quick path to a segmentation fault. So "obj == null" would either yield false or crash your program. I remember this, and I remember arguing for the current behavior many times (after having many many crashes) :) https://forum.dlang.org/post/fqlgah$15v2$1...@digitalmars.com Read that thread if you want to see the rationale. However, I'd argue it's still good to keep the error as there is literally no point to not using == null on class references vs. is null. You now would get the same result, but it's faster/cleaner. Actually, that runtime function has existed since before TDPL came out in 2010. It even shows the implementation of the free function opEquals (which at the time was in object_.d rather than object.d). I'm not even sure that the error message was added before the free function version of opEquals was. Maybe when that error message was first introduced, it avoided a segfault, but if so, it has been a _long_ time since that was the case. Some things in TDPL were forward-thinking. I remember Andrei fleshing out some of how the languages SHOULD behave in the forums or mailing lists for the purposes of writing TDPL even though it didn't yet behave that way. In fact, I'm almost positive the new object comparison function came as a result of TDPL (but I'm not 100% sure). Some of TDPL still has never been implemented. Long story short, don't date the existence of features in TDPL based on the publication :) In this case, for fun (what is wrong with me), I looked up the exact date it got added, and it was Feb 2010: https://github.com/dlang/druntime/commit/2dac6aa262309e75ad9b524cb4d1c3c1f0ecc2ae. TDPL came out in June 2010, so this feature does predate TDPL by a bit. In fact, through this exercise, I just noticed that the reason it returns auto instead of bool is to make sure it gets into the now defunct "generated" object.di file (https://github.com/dlang/druntime/pull/2190). It *is* faster to call "foo is null" than "foo == null", but I don't think that's particularly worth a compiler error. The compiler could just convert it to "is null" automatically in that case. It's not worth a compiler error if we didn't already have it, but I don't know that it's worth taking out. It's really what you should be doing, it's just that the penalty for not doing it isn't as severe as it used to be. One casualty of the current state of affairs is that no object may compare equal to null. And let's keep it that way! Of course, the most notable case where using == with null is a terrible idea is dynamic arrays, and that's the case where the compiler _doesn't_ complain. I use arr == null all the time. I'm perfectly fine with that, and understand what it means. Using == with null and arrays is always unclear about the programmer's intent and almost certainly wasn't what the programmer intended. I beg to differ. If the programmer cares about null, they should use is. If they care about lengnth, then that's what they should check. Checking null with == is just a huge code smell. IMO, doing anything based on the pointer of an array being null is a huge code smell. In which case, == null is perfectly acceptable. I'm comparing my array to an empty array. What is confusing about that? I actually hate using the pointer in any aspect -- an array is semantically equivalent to its elements, it's not important where it's allocated. The only place D forces me to care about the pointer is when I'm dealing with ranges. So, perhaps the compiler is being pedantic, but it's still telling you the right thing. It's just insisting about it in the case where it matters less while not complaining aobut it in the case where it really matters, which is dumb. So IMHO, if anything, adding an error message for the array case would make more sense than getting rid of the error with pointers and references. I hope this never happens. -Steve
Re: UFCS syntax I never saw before.
On Monday, 21 May 2018 at 11:38:12 UTC, SrMordred wrote: After all this time I saw this: writeln = iota = 5; what?? I never saw that before! This is interesting, there is something useful that i can do with this kind of call? What the hell is this? I don't figure out why are there so many syntax features? It doesn't make the language more easier. Is it any reason to support 2 additional cases of function calling? We have no literal for a tuple but we call a function via 3 cases. I'm frustrated. D has a lot cool features, I like D, but that is one of those which make me doubt the language future. I don't play a big role in the project but I talk to all my python co-workers: "You know... so there so cool language, it's called D. It can do this and that... It supports convenient static typing so you definitely know what happens in a piece of code.". But something like this make doubt about what happens in a piece of code. There is static typing so I know that type a variable has, but I look at a method calling and I ask: "Is it data field? No. Is it property? No. Is it method? No. It's UFCS! Okay." And now I see that UFCS can works the same way as a property!
Re: Mysql query result access by field name
On Sunday, 20 May 2018 at 16:08:03 UTC, ipkwena wrote: How does one access the columns fields in a Mysql query results by the column name. [...] Data f; f.name = to!string(allrows[0][0]); f.surname = to!string(allrows[0][1]); f.title = to!string(allrows[0][2]); I am using the mysql-native package or DB connectivity. According to the source code https://github.com/mysql-d/mysql-native/blob/master/source/mysql/result.d it should be possible to write the rowdata into the struct: Data f; allrows[0].toStruct (f); I haven't checked this.
assertNotThrown (and asserts in general)
I was interested by asserts and how the compiler uses them to optimize the code. So I looked at the compiler explorer to see how and found it, it doesn't. What I tried to do is turn a std.conv.to!ulong(byte) to a simple cast with the help of assertions. https://godbolt.org/g/4uckWU If there is an assert right before an if with the same condition, I think it should remove the compare and jump in release, but it doesn't. I'm assuming the compilers just aren't ready yet, but should be someday. At least that is what the documentation promises. More curious made me "assertNotThrown!ConvOverflowException(input.to!ubyte)" where it didn't even remove the assert code in release and produced the most inefficient assembly. Is that intended behaviour? In my opinion that greatly limits the usabilty of it.
Re: UFCS syntax I never saw before.
Right, so this should´n be working I think.
struct SomeStruct
{
void foo(int);
}
SomeStruct s;
s.foo = 10;
I thought that only with @property this will work.
Re: Copying an std.stdio.File into another one
On 21/05/2018 11:50 PM, 0xEAB wrote:
What's the correct way to copy a `File` into another one in D?
If `LockingTextReader` wasn't undocumented, I'd have gone for that
approach:
import std.algorithm.mutation : copy;
import std.stdio : File, LockingTextReader;
void main()
{
auto a = File("a.txt", "r");
auto b = File("b.txt", "w");
a.LockingTextReader.copy(b.lockingTextWriter);
}
Side-note:
Although the example code suggest otherwise, I'm not talking about
copying actual files on disk. In such a case I'd just use
`std.file.copy` ;)
So, just imagine `a` were `stdout` of process pipe.
Kind regards,
Elias
I would probably start by reading it byChunk and writing it out as a
rawWrite or something along those lines. That way its using a nice
buffer (to limit memory usage).
Copying an std.stdio.File into another one
What's the correct way to copy a `File` into another one in D?
If `LockingTextReader` wasn't undocumented, I'd have gone for
that approach:
import std.algorithm.mutation : copy;
import std.stdio : File, LockingTextReader;
void main()
{
auto a = File("a.txt", "r");
auto b = File("b.txt", "w");
a.LockingTextReader.copy(b.lockingTextWriter);
}
Side-note:
Although the example code suggest otherwise, I'm not talking
about copying actual files on disk. In such a case I'd just use
`std.file.copy` ;)
So, just imagine `a` were `stdout` of process pipe.
Kind regards,
Elias
Re: UFCS syntax I never saw before.
On Monday, 21 May 2018 at 11:38:12 UTC, SrMordred wrote:
After all this time I saw this:
writeln = iota = 5;
what??
I never saw that before!
This is interesting, there is something useful that i can do
with this kind of call?
I probably wouldn't use that. That wasn't what it was intended
for and it's not really UFCS. It's was meant for properties that
are defined as functions.
struct SomeStruct
{
void foo(int);
}
SomeStruct s;
s.foo = 10;
It's kind of horrible syntax for what it is doing, where it isn't
obvious what is happening. Writeln and iota aren't setting
anything, they are just function calls that happen to be able to
take one parameter.
UFCS syntax I never saw before.
After all this time I saw this: writeln = iota = 5; what?? I never saw that before! This is interesting, there is something useful that i can do with this kind of call?
