function to tell if a string could be an identifier
Excepting keywords (though it's OK if that's checked also), is there a function in Phobos or druntime to tell me if a string could be a valid identifier? I can write my own, but I was hoping this was already done. -Steve
Re: Is there a way to return an lvalue and also an rvalue from the same member function?
On 9/20/20 11:52 AM, realhet wrote:
On Sunday, 20 September 2020 at 14:54:09 UTC, Steven Schveighoffer wrote:
On 9/20/20 9:30 AM, realhet wrote:
ref inout(int) x() inout { return array[0]; }
This doesn't work when I type:
v.x++;
It should, as long as v is mutable.
I want to make a similar type like the GLSL vectors. Where the following
thing is valid:
vec4 a, b;
a.yzw = b.xzy;
This should be straight-up opDispatch I would think. You might need a
helper return that reroutes the correct items.
The only thing I don't want to implement from the GLSL spec is those
non-contigous swizzle assignments like:
a.zyx = vec3(1,2,3) ***
but
a.xyz = vec3(1,2,3) should work.
What you could do, in this case, is make your return type either a
helper type that uses a slice of the original, or one that contains a
copy of the data in the right order, but is not assignable.
*** maybe with a struct that refers to the original vector and the
swizzle code it could be also possible. :D
Yeah, I think this might work.
-Steve
Re: Building LDC runtime for a microcontroller
On 9/20/20 10:51 AM, Dylan Graham wrote: On Saturday, 19 September 2020 at 20:39:38 UTC, aberba wrote: Do you attend our monthly D online meetups? We have monthly online meetups? I would love to join of course! Happening next weekend! https://forum.dlang.org/post/rjjcl4$30sm$1...@digitalmars.com Would love to hear about your work! -Steve
Re: Is there a way to return an lvalue and also an rvalue from the same member function?
On 9/20/20 9:30 AM, realhet wrote:
Hi,
struct S{
int[2] array;
ref x() { return array[0]; }
auto x() const { return array[0]; }
}
If there a way to write the function 'x' into one function, not 2
overloads.
I tried auto/const/ref mindlessly :D, also remembered 'inout', but
obviously those weren't solve the problem.
Your original code is an odd situation -- you want to return by ref if
it's mutable, but not if it's const?
Why not return by ref always, and just forward the constancy? This is
what inout is made to do:
ref inout(int) x() inout { return array[0]; }
(This is going to be a swizzling 'system' that mimics GLSL, later I will
make a template that takes 'x'as a template parameter, just wondering
that the support for const and non-cons can be done easier.)
If you want to differ behavior by const, but write one function, you can
use a `this` template parameter. But without seeing your real use case,
you might end up writing the same amount of code.
-Steve
Re: vibe.d: How to get the conent of a file upload ?
On 9/19/20 3:36 PM, IGotD- wrote: On Saturday, 19 September 2020 at 19:27:40 UTC, Steven Schveighoffer wrote: I used Kai's book, and yeah, you have to do things the vibe way. But most web frameworks are that way I think. Do you have a reference to this book (web link, ISBN)? Sure: https://www.packtpub.com/product/d-web-development/9781785288890 It might be a bit dated, but most of the concepts are the same. -Steve
Re: vibe.d: How to get the conent of a file upload ?
On 9/19/20 6:59 AM, wjoe wrote: Handling file uploads is one example, another is command line arguments. The presumption here is there is vibe and there can only be vibe. It takes them from Runtime.args. Duh? This didn't make sense to me until I saw example where the initialization of vibe was done in a module constructor. This used to be the expected way to set up vibe (using module constructors). And vibe would provide its own main function. I *think* the idea was to allow registration of different handlers in their respective modules. Nowadays, you just run the setup wherever you want (I do everything in main). By using vibe I feel like I need to bend myself like a snake and jump through the hoops of vibe's one way to do it. You save a lot of time here and there and then you lose half a day because of some stupid road block like the above. I used Kai's book, and yeah, you have to do things the vibe way. But most web frameworks are that way I think. I recommend getting the book if you plan on doing a lot with vibe.d Where vibe really shines are the diet templates and the web interface stuff. To not have to handle anything from forms, or query parameters, and just write normal D functions is really great. You can even do a lot of authentication and stuff using UDAs. It helps you write consistent web applications. And I LOVE diet templates. So much less verbose than actual HTML. I have a hard time writing actual HTML now. When you want to do stuff manually, I think it's not as well supported. -Steve
Re: vibe.d: How to get the conent of a file upload ?
On 9/19/20 7:15 AM, ikod wrote: On Saturday, 19 September 2020 at 11:11:21 UTC, ikod wrote: On Friday, 18 September 2020 at 13:13:16 UTC, wjoe wrote: On Friday, 18 September 2020 at 12:58:29 UTC, Steven Schveighoffer wrote: On 9/18/20 8:39 AM, Steven Schveighoffer wrote: But again, solved with an enhancement that allows you to process the data in your code. I'll file the enhancement request for you, as I think it's a nice addition. https://github.com/vibe-d/vibe.d/issues/2478 Awesome! Thanks a ton :) My preferable way to handle such thing is: convert incoming data into input range of immutable(ubyte)[] and let user to consume this range (and handle it as it wish - transform, store in memory as is, or dump to disk) sorry, this looks exactly like it described in proposal (and this is how requests works). You mean for each file, right? Yeah, that is the proposal, though it uses Vibe's io type (necessary I think). -Steve
Re: Question about linker errors when using slices
On 9/18/20 10:45 PM, tspike wrote: If you only compile platform.d, the linker will complain about “undefined references.” This is true when using dmd and gdc, though platform.d compiles just fine when using ldc. But the file only fails to compile when the “items” member of AppData is a slice; if “items” is an int* platform.d will compile. The linker spits the following: platform.o:(.data._D22TypeInfo_S3app7AppData6__initZ+0x30): undefined reference to `_D3app7AppData9__xtoHashFNbNeKxSQBeQBdZm' platform.o:(.data._D22TypeInfo_S3app7AppData6__initZ+0x38): undefined reference to `_D3app7AppData11__xopEqualsFKxSQBdQBcKxQjZb' I was just wondering if anyone knows if this behavior is expected or if this is a compiler bug. Thank you in advance for your time! On one hand, I don't necessarily expect it. These are symbols that are used to build an appropriate TypeInfo instance. I wouldn't expect it, because you aren't using TypeInfo anywhere. However, it does happen quite a bit, because the conditions under which D generates or expects a generated TypeInfo are somewhat obscure and not documented. It's probably why LDC works and the others don't. I hope Andrei's recent push to demystify TypeInfo stuff makes this a lot more tractable. The answer is -- just build with all the files. The linker should throw out stuff that isn't needed. PS: I hope this is the right sub-forum for asking this sort of question! Of course! -Steve
Re: dub: Is it possible to have a library target and depend on it in the same dub config?
On 9/18/20 7:38 AM, wjoe wrote:
Something like this:
configuration "lib" {
targetType "dynamicLibrary"
sourceDir "source/lib/"
}
configuration "app" {
targetType "executable"
sourceFiles "source/app.d"
linkWith "lib"
}
I found subConfiguration in the docs but that seems to be related to
external dependencies.
app.d merely provides a CLI to the library as well as an option to run
as a server. I don't want to have the overhead of an entire git repo and
such for something that is a single file and also closely related to the
library.
There are other options.
for instance dub (the project) has a library and an application. the
config looks like this:
configuration "application" {
targetType "executable"
mainSourceFile "source/app.d"
libs "curl"
versions "DubUseCurl" "DubApplication"
}
configuration "library" {
targetType "library"
excludedSourceFiles "source/app.d"
copyFiles "bin/libcurl.dll" "bin/libeay32.dll" "bin/ssleay32.dll"
platform="windows"
versions "DubUseCurl"
}
You can also build a subproject in the same repository. In that case,
you would probably want the app to be the main project, and you then
depend on the library project via "foo:lib"
-Steve
Re: DDoc generation
On 9/18/20 7:41 AM, Russel Winder wrote: Hi, I am trying to get to grips with DDoc for documenting an application. Getting the individual module HTML files seems to be the easy bit. The question is how to get an index.html (or equivalent) so as to have an application level entry point to the generated documentation. You can make explicit ddoc files, and compile those along with your application. https://dlang.org/spec/ddoc.html#using_ddoc_for_other_documentation -Steve
Re: vibe.d: How to get the conent of a file upload ?
On 9/18/20 8:39 AM, Steven Schveighoffer wrote: But again, solved with an enhancement that allows you to process the data in your code. I'll file the enhancement request for you, as I think it's a nice addition. https://github.com/vibe-d/vibe.d/issues/2478 -Steve
Re: vibe.d: How to get the conent of a file upload ?
On 9/17/20 8:07 PM, wjoe wrote: Not a reply to this post in particular but to all the ones I've read so far. If I understand correctly. Vibe parses the form data and writes all files to disk. Where to ? See the code here: https://github.com/vibe-d/vibe.d/blob/ebebfa827f568cc9bced4bec2b66edc043a8adf7/inet/vibe/inet/webform.d#L311 Can I configure it ? I don't want libraries to just write data to my file systems without me setting this up. Nowhere did I find this behavior described in the docs. No, not at the moment. Which is why I was saying, it could be an enhancement request to vibe. And if not, how is data processed with a 10mb file upload followed by a few number fields ? All the data is processed before the accessor to the form data or the file data. It HAS to be this way, as the data is still on the incoming network socket. It needs to read all of the file data to get to the other data fields, doesn't it ? Yes I'm sorry this is completely counter intuitive. I can understand the memory/security risks and all but I have no intention to hack, DOS or however else disrupt my private server in my private network with garbage data. I just want to get the data in a byte[]. Again, enhancement request needed. The code currently is hard-coded to write to disk. Why does the lib not simply reject files that are unreasonably (configurable) big ? If you had 1000 requests being processed simultaneously, and each of those requests provided 10MB files, then you now need potentially 10GB of RAM to hold those requests. This doesn't scale when the application is unknown to vibe. But again, solved with an enhancement that allows you to process the data in your code. I'll file the enhancement request for you, as I think it's a nice addition. Writing files to disk in order to then needing to copy them somewhere else or to read them back into memory for further processing sounds, above all else, incredibly inefficient. Agreed. In my case, it was an actual copy, as the location of the stored data was on a different filesystem than the temporary files. I might not even want to keep the file and drop it. Yep, it's a waste in that case. I guess it's no problem to parse the data myself, but then what's the point in using a framework ? Agreed. Are there other frameworks besides vibe that can do what I want ? In D, I'm not sure what the other frameworks do. I believe there are others if you search on code.dlang.org, you should be able to find some. I'm sorry for the rant, developing this kind of software is a pain in the drain and stresses me out to no end. It sucked hard in the past with php and decades later with python, ruby, D, you name it it still sucks ;) web development sucks in general ;) Yet, we all still do it. -Steve
Re: vibe.d: How to get the conent of a file upload ?
On 9/17/20 6:13 PM, aberba wrote:
On Thursday, 17 September 2020 at 21:57:37 UTC, Steven Schveighoffer wrote:
On 9/17/20 1:08 PM, wjoe wrote:
[...]
the `files` property actually does the processing only when you call it.
If you access the `bodyReader` property directly, you can process that
data yourself. You can even register a web interface function with an
`InputStream` parameter type, and it will be bound to the body data.
I'm not sure I understand how to do this and parser the files in memory.
So an HTTP request with form data will come in with the headers parsed,
but the data is still on the network stream.
The first time you access `files`, it processes the stream data, and
splits it into form data and file data, saves the files, and then gives
you back the file dictionary so you can use them.
If instead, you access `bodyReader`, YOU get to process the form data
and file data.
I've done this with my REST interface, though that's not form data.
That's not a great API, though. I would love to see vibe.d allow a
direct call to vibe.inet.webform.parseFormData with a specific handler
for files and form data.
Can we file an issue for this? Because I'm very interested in having
this resolved
You can always file an issue! https://github.com/vibe-d/vibe.d/issues
There may already be one in there.
There's potential to results in out of memory condition. Its a know
issues. A complete parser (like multer in nodejs) allowance you to limit
file size as well for error handling.
Meh, this is D :) we should be able to just process the data and do
whatever we want with it. What I would like to see is vibe provide the
parsing of form data, and just give me the data as it comes (kind of
like a SAX parser). Maybe just a property in the HTTPServerRequest that
I can set that says "use this callback when you get Form File data".
I've done this with my REST interface, though that's not form data.
Can you share your code for this?
Heh, this is not form data, it's just file data, raw on the stream. So I
have a function like:
```
class FileRestImpl
{
@path(":cat/:id/:uuid/upload")
@getAuth
void postUpload(HTTPServerResponse res, string _cat, int _id,
string _uuid, InputStream stream, Nullable!string md5sum, NRMAuthInfo
_authInfo)
{
...
}
}
```
You can see, I take an InputStream as a parameter -- the data comes in
there. I just read it and save it to a file (in the correct location)
anyway, verifying the md5sum is valid.
-Steve
Re: enum and const or immutable ‘variable’ whose value is known at compile time
On 9/17/20 9:13 AM, Simen Kjærås wrote: To be clear: I don't mind 'enum' being used this way, but if I were to do things over again, I would have used 'alias'. fun fact: for a (very) brief time, D had a `manifest` keyword that did exactly what enum does in this instance (not even sure it made it into a release). enum is a head scratcher of a name, for sure. But it works out just fine once you get used to it. I think of it as "compile time". To be honest, for what it does, enum is a very poor name. But because it's consistent, it works. -Steve
Re: vibe.d: How to get the conent of a file upload ?
On 9/17/20 1:08 PM, wjoe wrote:
Every post or example I found copies the file, like your code does, too.
Why is that ? The content of the file upload is embedded in the form
data. There's no need for temporary files or copying at all.
On top of that, if I upload a file to a server which is on a different
PC on a different file system, how am I supposed to read the file from
disk on a remote file system ?
This makes no sense.
What I want is something like this:
~$ cat /var/log/temperatures.log
temp_1=22;temp_2=28
temp_1=21;temp_2=25
~$ curl -F "temperature_log=@/var/log/temperatures.log"
192.168.1.1:20222/temperature_upload
~$ nc -l 127.0.0.1 20222
POST /temperature_upload HTTP/1.1
Host: 192.168.1.1:20222
User-Agent: curl/7.72.0
Accept: */*
Content-Length: 245
Content-Type: multipart/form-data;
boundary=c73c71472ff9e7d5
--c73c71472ff9e7d5
Content-Disposition: form-data; name="temperature_log";
filename="/var/log/temperatures.log"
Content-Type: application/octet-stream
temp_1=22;temp_2=28
temp_1=21;temp_2=25
--c73c71472ff9e7d5--
void upload(HttpRequest.. req, blah)
{
auto file = "temperature_log" in req.files;
if (file) {
string temp_data_raw = file.data;
assert ("temp_1=22;temp_2=28\ntemp_1=21;temp_2=25" ==
temp_data_raw);
}
}
the `files` property actually does the processing only when you call it.
If you access the `bodyReader` property directly, you can process that
data yourself. You can even register a web interface function with an
`InputStream` parameter type, and it will be bound to the body data.
I've done this with my REST interface, though that's not form data.
That's not a great API, though. I would love to see vibe.d allow a
direct call to vibe.inet.webform.parseFormData with a specific handler
for files and form data.
I think you can agree that it's not feasible to store arbitrary sized
file contents in memory. But it certainly can provide a mechanism to
handle it as it's read.
-Steve
Re: Why is BOM required to use unicode in tokens?
On 9/15/20 8:10 PM, James Blachly wrote: On 9/15/20 10:59 AM, Steven Schveighoffer wrote: Thanks to Paul, Jon, Dominikus and H.S. for thoughtful responses. What will it take (i.e. order of difficulty) to get this fixed -- will merely a bug report (and PR, not sure if I can tackle or not) do it, or will this require more in-depth discussion with compiler maintainers? I'm thinking your issue will not be fixed (just like we don't allow $abc to be an identifier). But the spec can be fixed to refer to the correct standards. Steve: It sounds as if the spec is correct but the glyph (codepoint?) range is outdated. If this is the case, it would be a worthwhile update. Do you really think it would be rejected out of hand? I don't really know the answer, as I'm not a unicode expert. Someone should verify that the character you want to use for a symbol name is actually considered a letter or not. Using phobos to prove this is kind of self-defeating, as I'm pretty sure it would be in league with DMD if there is a bug. But if it's not a letter, then it would take more than just updating the range. It would be a change in the philosophy of what constitutes an identifier name. -Steve
Re: Why is BOM required to use unicode in tokens?
On 9/15/20 10:18 AM, James Blachly wrote: On 9/15/20 4:36 AM, Dominikus Dittes Scherkl wrote: On Tuesday, 15 September 2020 at 06:49:08 UTC, Jon Degenhardt wrote: On Tuesday, 15 September 2020 at 02:23:31 UTC, Paul Backus wrote: Identifiers start with a letter, _, or universal alpha, and are followed by any number of letters, _, digits, or universal alphas. Universal alphas are as defined in ISO/IEC 9899:1999(E) Appendix D of the C99 Standard. I was unable to find the definition of a "universal alpha", or whether that includes non-ascii alphabetic characters. ISO/IEC 9899:1999 (E) Annex D Universal character names for identifiers - --- This is outdated to the brim. Also it doesn't allow for letter-like symbols (which is debatable, but especially the mathematical ones like double-struck letters are intended for such use). Instead of some old C-Standard, D should better rely directly on the properties from UnicodeData.txt, which is updated with every new unicode version. Thanks to Paul, Jon, Dominikus and H.S. for thoughtful responses. What will it take (i.e. order of difficulty) to get this fixed -- will merely a bug report (and PR, not sure if I can tackle or not) do it, or will this require more in-depth discussion with compiler maintainers? I'm thinking your issue will not be fixed (just like we don't allow $abc to be an identifier). But the spec can be fixed to refer to the correct standards. -Steve
Re: Get enum value name as string at compile time?
On 9/14/20 2:25 AM, Simen Kjærås wrote:
On Monday, 14 September 2020 at 03:48:51 UTC, Steven Schveighoffer wrote:
Consider the enum:
enum Foo { a, b }
Foo.a.stringof => "a"
enum x = Foo.a;
x.stringof => "cast(Foo)0"
Is there another way I can take an enum value that's known at compile
time (but not the actual identifier), and get the name of it? I know I
can use a switch, or to!string. But I was hoping this was easy for the
compiler to figure out some way without involving CTFE.
It is a bit weird that x.stringof doesn't simply return the name like
Foo.a.stringof does. Anyways, this works:
template enumName(alias a) {
import std.meta : staticIndexOf, staticMap;
alias T = typeof(a);
enum getValue(string name) = __traits(getMember, T, name);
alias enumValues = staticMap!(getValue, __traits(allMembers, T));
enum enumName = __traits(allMembers, T)[staticIndexOf!(a,
enumValues)];
}
enum Foo { a = 2, b = 19 }
enum x = Foo.a;
pragma(msg, enumName!x); // "a"
Thanks.
I never considered doing something like that. Not sure I like that
better than the CTFE version. I just punted and used to!string in my
code anyway.
A CTFE linear search in a compile-time array probably wouldn't be too
bad, especially when the list of elements is not long.
Again, we could use that ... DIP to make things a lot less complex.
-Steve
Get enum value name as string at compile time?
Consider the enum:
enum Foo { a, b }
Foo.a.stringof => "a"
enum x = Foo.a;
x.stringof => "cast(Foo)0"
Is there another way I can take an enum value that's known at compile
time (but not the actual identifier), and get the name of it? I know I
can use a switch, or to!string. But I was hoping this was easy for the
compiler to figure out some way without involving CTFE.
-Steve
Re: Call C variadic function from D variadic function
On 9/13/20 2:35 PM, Paul Backus wrote:
On Sunday, 13 September 2020 at 17:23:42 UTC, Steven Schveighoffer wrote:
On 9/13/20 12:55 PM, James Blachly wrote:
```
/// Add a single line to an existing header
auto addLine(T...)(RecordType type, T kvargs)
if(kvargs.length > 0 && isSomeString!(T[0]))
{
static assert (kvargs.length %2 == 0); // K-V pairs =>
even number of variadic args
string varargMagic(size_t len)
{
string args = "sam_hdr_add_line(this.h, type.ptr, ";
for(int i=0; iInterestingly, compilation fails if the mixin consists only of the
comma-separated parameters ("Comma expression" [1])
Question:
If a variadic template, despite presenting to the user a "dynamic
array", MUST know its parameter list at compile-time, is there a way
(other than with mixins as shown) to pass this parameter list to
extern(C) linkage function with variadic parameters?
Was just talking about this exact problem with Adam Ruppe.
Unfortunately, because the parameters are an expression tuple, and not
a compile-time tuple, you can't use stuff like staticMap.
You actually can, if you define the right kind of helper function:
/// Add a single line to an existing header
auto addLine(T...)(RecordType type, T kvargs)
if(kvargs.length > 0 && isSomeString!(T[0]))
{
static assert (kvargs.length %2 == 0); // K-V pairs => even
number of variadic args
immtuable(char)* argToStringz(alias arg)()
{
return toStringz(arg);
}
return sam_hdr_add_line(this.h, this.ptr,
staticMap!(argToStringz, kvargs), null);
}
The clever trick here is that, because of optional parentheses [1],
`argToStringz!(kvargs[i])` can be interpreted either as the name of a
function or a function call, depending on the context it appears in.
That's cool. And horrific at the same time :) I mean the templates that
you have to instantiate for this...
I would prefer the mixin solution, even though it's uglier. I think
something that abstracts that out would be a nice thing to have for
std.meta.
-Steve
Re: Call C variadic function from D variadic function
On 9/13/20 12:55 PM, James Blachly wrote:
Summary:
Can a typesafe D variadic function, or D variadic template pass its
parameters to a C variadic function?
Background:
I maintain a library binding [0] to htslib, a high-performance and very
widely used C library for high-throughput sequencing (hts) data files.
We use this internally and haven't polished it for a release on the
announce forum or biomed twitter, etc. yet.
In the course of upgrading it to support the latest API/ABI (htslib-1.10
/ so.3), I need to add support for several new functions.
One of these is a variadic function with the signature:
`int sam_hdr_add_line(sam_hdr_t *h, const char *type, ...);`
Of course, we can call this directly from D with hardcoded parameters.
However, one of the focuses of our library is "bindings + wrappers" to
make this feel more like native D. Thus, we want a variadic function to
which we may pass D strings (it is also a struct member function).
With help from Herringway on IRC, we came up with a solution using mixin:
```
/// Add a single line to an existing header
auto addLine(T...)(RecordType type, T kvargs)
if(kvargs.length > 0 && isSomeString!(T[0]))
{
static assert (kvargs.length %2 == 0); // K-V pairs => even
number of variadic args
string varargMagic(size_t len)
{
string args = "sam_hdr_add_line(this.h, type.ptr, ";
for(int i=0; iInterestingly, compilation fails if the mixin consists only of the
comma-separated parameters ("Comma expression" [1])
Question:
If a variadic template, despite presenting to the user a "dynamic
array", MUST know its parameter list at compile-time, is there a way
(other than with mixins as shown) to pass this parameter list to
extern(C) linkage function with variadic parameters?
Was just talking about this exact problem with Adam Ruppe.
Unfortunately, because the parameters are an expression tuple, and not a
compile-time tuple, you can't use stuff like staticMap.
Manu's ... dip would be perfect for this:
return sam_hdr_add_line(this.h, this.ptr, toStringz(kvargs)..., null);
I think the only way it works today is if you use the mixin technique.
-Steve
Re: Initialize to None
On 9/5/20 11:42 PM, N.S. wrote:
I'd like to check whether a variable is initialized or not. And I'd also
like to uninitialize a variable that is already initialized. Thanks!
int x = void;
if (x == void)
There isn't a way to check this.
{
writeln("x not initialized");
}
else
{
// OK, use x
}
// Uninitialize x
x = void;
When you set the initial value of x to void, it means "don't do
anything, let whatever data is already on the stack be the value of x".
So "uninitialize" doesn't really mean anything. It would technically
mean "do nothing".
-Steve
Re: I think Associative Array should throw Exception
On 9/4/20 1:48 AM, Jesse Phillips wrote:
On Thursday, 3 September 2020 at 15:12:14 UTC, Steven Schveighoffer wrote:
int[int] aa;
aa[4] = 5;
auto b = aa[4];
How is this code broken? It's valid, will never throw, and there's no
reason that we should break it by adding an exception into the mix.
int foo() nothrow {
return "1".to!int;
}
The following code is valid, will never throw, why does the compiler
prevent it?
You are still missing the point ;)
Your example doesn't compile today. Mine does. It's not a question of
which way is better, but that we already have code that depends on the
chosen solution, and changing now means breaking all such existing code.
My point of bringing up the example is that your assertion that "it is
already broken" isn't true.
To put it another way, if the above to!int call compiled, and we
switched to exceptions, it would be the same problem, even if the right
choice is to use Exceptions.
-Steve
Re: I think Associative Array should throw Exception
On 9/3/20 10:43 AM, Jesse Phillips wrote:
On Tuesday, 1 September 2020 at 18:55:20 UTC, Steven Schveighoffer wrote:
On 9/1/20 2:20 PM, Jesse Phillips wrote:
Using RangeError is nice as it allows code to use array index inside
`nothrow.`
This is the big sticking point -- code that is nothrow would no longer
be able to use AAs. It makes the idea, unfortunately, a non-starter.
What is wrong with using `in`? I use this mostly:
if(auto v = key in aa) { /* use v */ }
I think that actually might be my point. If you need nothrow then this
is what you need to do.
For breaking nothrow code using the [] syntax, I'd say it is already
broken because the behavior is to throw and the above is how you would
check that it won't.
int[int] aa;
aa[4] = 5;
auto b = aa[4];
How is this code broken? It's valid, will never throw, and there's no
reason that we should break it by adding an exception into the mix.
The issue is, associative arrays throw an "uncatchable" error. Meaning
code is written to catch the error (because it works). And correctly
written `nothrow` code needs to use `in` to be properly nothrow.
The big issue is -- is accessing an invalid index a programming error or
an environmental error? The answer is -- it depends. D has declared, if
you use the indexing syntax, then it's a programming error. If you want
it not to be a programming error, you use the key in aa syntax, and
handle it.
The other thing you can do is use a different type, if you don't want to
deal with the verbose syntax, but still want to catch environmental
errors. A wrapper type is possible.
-Steve
Re: Annotating SortedRange.empty const: Best way to auto-deduce annotations?
On 9/2/20 5:23 PM, SimonN wrote: Hi, About this issue in Phobos: https://issues.dlang.org/show_bug.cgi?id=21216 SortedRange.empty should be const, .front should be inout Just adding const/inout to SortedRange's methods won't be enough; if we add const/inout here, then many other Phobos ranges need to become const/inout-correct to keep the tests passing. Before I dive deeper into annotating their methods, I would like to verify my assumptions on how template function attribute deduction works: 1) I know that templated functions deduce their own attributes on instantiation. But front()/empty() are non-templated methods within the templated struct SortedRange. Will attribute deduction happen here? mutability attributes are not deducted for templates. wrapping ranges such as SortedRange can't really specify const or inout on their methods via introspection of the underlying range. What they can do is template the `this` parameter. Then if the underlying range supports calling that way, it will work, otherwise it won't. using `template this` should be compatible with the existing code I would think. -Steve
Re: Bug in import(...) on Windows?
On 9/2/20 4:48 PM, Andrey Zherikov wrote:
On Wednesday, 2 September 2020 at 20:23:15 UTC, Steven Schveighoffer wrote:
What I'm wondering is if it needs to be ./file instead of .\file. Can
you hard code that and see if it works?
This actually works:
pragma(msg, import("file"));
pragma(msg, buildPath(".", "file"));
pragma(msg, import("./file"));
Result on Ubuntu:
===
hello
../file
hello
===
Result on Windows:
===
hello
..\file
hello
===
This seems weird that I can't use std.path functions or use valid
"foo\bar" paths on Windows.
I don't know why it wouldn't work with native paths on Windows. But I'm
not a DMD maintainer, so I don't know where to look in the code to see
why it doesn't work.
It might be worth filing a bug. https://issues.dlang.org
-Steve
Re: Bug in import(...) on Windows?
On 9/2/20 1:47 PM, Adam D. Ruppe wrote:
On Wednesday, 2 September 2020 at 17:39:04 UTC, Andrey Zherikov wrote:
Is this a bug in dmd?
I think it is an old bug filed (I can't find it though) about
inconsistent platform behavior but it is allowed by spec for the
compiler to reject any path components.
import("") is supposed to just give a filename, no directory path.
See: https://dlang.org/spec/expression.html#import_expressions
"Implementations may restrict the file name in order to avoid directory
traversal security vulnerabilities. A possible restriction might be to
disallow any path components in the file name."
Is this the problem though? It works on DMD Linux, which shares the same
front end.
What I'm wondering is if it needs to be ./file instead of .\file. Can
you hard code that and see if it works?
FYI, I know vibe diet templates DEPEND on this behavior, and I'd be
surprised if it doesn't work at all on Windows.
-Steve
Re: How to create compile-time container?
On 9/2/20 5:56 AM, Andrey Zherikov wrote:
==
Everything works well until I have included scripts in subdirectories:
├── dir1
│ ├── dir2
│ │ └── script
│ └── script
└── script
Content:
== script
msg hello
include dir1/script
== dir1/script
msg hello from dir1
include dir2/script
== dir1/dir2/script
msg hello from dir1/dir2
==
Compilation fails with "Error: file `"dir2/script"` cannot be found or
not in a path specified with `-J`" (I used simple dmd -J. -run parser.d)
which is expected because parse* functions do not track the directory
where the script is located.
In this simple example the issue can be fixed by passing path to script
as a parameter to parseScript function. But this doesn't seem to be
flexible and extendable solution because there can be other commands
that might call parseFile indirectly (they can even be in other modules).
Theoretically this can be solved by doing something like this but it
doesn't work because "static variable `paths` cannot be read at compile
time":
==
string[] paths;
void parseFile(string file)()
{
enum path = paths.length > 0 ? buildPath(paths[$-1],
file.dirName()) : file.dirName();
paths ~= path;
scope(exit) paths = paths[0..$-1];
enum script = import(buildPath(path, file));
mixin(parseScript(script));
}
==
Note that the whole point is to do this parsing at compile time.
OK, NOW I see where your code is coming from. The issue is that you need
the directory of the script imported to be the "local directory". Your
solution will not work -- you can't mixin code that is not available at
compile time.
Here is what I would do instead:
string parseScript(string filename, string script)
{
string code;
string base = dirName(filename);
if(base[$-1] != '/') base ~= '/';
foreach(line; script.lineSplitter())
{
auto idx = line.indexOf(' ');
switch(line[0..idx])
{
case "msg":
code ~= "writeln(\"" ~ line[idx+1..$] ~ "\");";
break;
case "include":
{
code ~= `parseFile!"`;
string importfile = line[idx+1 .. $];
if(!importfile.startsWith('/')) // relative path
code ~= base;
code ~= importfile ~ `";`;
break;
}
default: break;
}
}
return code;
}
And pass the filename to this function in addition to the script source.
-Steve
Re: I think Associative Array should throw Exception
On 9/1/20 10:46 PM, James Blachly wrote: On 9/1/20 2:55 PM, Steven Schveighoffer wrote: On 9/1/20 2:20 PM, Jesse Phillips wrote: Using RangeError is nice as it allows code to use array index inside `nothrow.` This is the big sticking point -- code that is nothrow would no longer be able to use AAs. It makes the idea, unfortunately, a non-starter. Steve, are there not several (probably better, faster) alternatives to the built-in AA that are nothrow? I think a nice way to look at the built-in AA is an easy default for quick scripts, new users, etc., much like the default of `throw` status of a function or code block. Advanced users, (i.e. those using nothrow annotation) could select a more efficient AA implementation anyway. The problem is not the requirement but the resulting code breakage if you change it now. -Steve
Re: How to create compile-time container?
On 9/1/20 3:09 PM, Andrey Zherikov wrote: Unfortunately this won't work if there is a function 'bar' in different module that calls 'foo': You should post a full example you expect to work or not work, then we can discuss. I think it should work (I've tried it), but there are several problems that could possibly happen with your code, and it's hard to tell what you mean by "won't work" with the incomplete code that you posted. -Steve
Re: How to create compile-time container?
On 9/1/20 2:19 PM, Andrey Zherikov wrote:
On Monday, 31 August 2020 at 20:44:16 UTC, Adam D. Ruppe wrote:
On Monday, 31 August 2020 at 20:39:10 UTC, Andrey Zherikov wrote:
How can I do that?
You can use a normal string[] BUT it is only allowed to be modified
inside its own function.
Then you assign that function to an enum or whatever.
string[] ctGenerate() {
string[] list;
list ~= "stuff";
return list;
}
enum list = ctGenerate();
That's all allowed. But CTFE is now allowed to read or modify anything
outside its own function; you can't have two separate function calls
build up a shared list (unless you can somehow call them both together
like `enum list = ctGenerate() ~ other_thing();`
The thing I'm trying to implement is: I have a function foo(string s)()
and some "state"; this function should override this "state" (using "s"
param) for all code within this function (note that code can execute
other modules that can refer to the same "state"). The problem is that I
need this overridden "state" to be compile-time constant to be used in
mixin. Any ideas how I can do this?
string overrideState(string s)
{
// work your magic here, it's normal D code!
}
void foo(string s)()
{
mixin(overrideState(s));
}
-Steve
Re: I think Associative Array should throw Exception
On 9/1/20 2:20 PM, Jesse Phillips wrote:
Using RangeError is nice as it allows code to use array index inside
`nothrow.`
This is the big sticking point -- code that is nothrow would no longer
be able to use AAs. It makes the idea, unfortunately, a non-starter.
What is wrong with using `in`? I use this mostly:
if(auto v = key in aa) { /* use v */ }
Note, that in certain cases, I want to turn *normal* array access errors
into exceptions, because I always want bounds checking on, but I don't
want a (caught) programming error to bring down my whole vibe.d server.
So I created a simple wrapper around arrays which throws exceptions on
out-of-bounds access. You could do a similar thing with AAs. It's just
that the declaration syntax isn't as nice.
-Steve
Re: Template argument deduction fails with alias
On 8/31/20 9:11 PM, Ben Jones wrote:
I have an alias that looks like
static if(...){
alias AliasType = SumType!(...);
}
which I use in a template constraint for a function template:
bool func(T: AliasType!Args, Args...)(T t){ ... }
When I try to call func with an AliasType object, the argument deduction
fails with a message saying that the argument type (a SumType) doesn't
match the template constraint (an AliasType)
Things do work if I change the template constraint to be a SumType
rather an an AliasType
Is there a workaround to this? Here's a complete example:
https://run.dlang.io/is/buRGTs
Very old enhancement request (I doubt this will ever happen):
https://issues.dlang.org/show_bug.cgi?id=1807
-Steve
Re: How do I convert an ISO 8601 datetime into a unix timestamp - at compile-time?
On 8/27/20 9:54 PM, Andrej Mitrovic wrote:
-
import std.datetime;
void main ()
{
static time =
SysTime(DateTime.fromISOString("20220101T00")).toUnixTime;
}
-
-
/Library/D/dmd/src/phobos/std/concurrency.d(2574): Error: static
variable lock cannot be read at compile time
/Library/D/dmd/src/phobos/std/concurrency.d(2574): called from
here: atomicLoad(lock)
/Library/D/dmd/src/phobos/std/concurrency.d(2600): called from
here: initOnceLock()
/Library/D/dmd/src/phobos/std/concurrency.d(2600): called from
here: initOnce(delegate shared(bool)() pure @nogc @safe => init(),
initOnceLock())
/Library/D/dmd/src/phobos/std/datetime/timezone.d(1106): called from
here: initOnce(delegate shared(bool)() pure nothrow @nogc @safe =>
(*function () nothrow @nogc @safe => true)())
/Library/D/dmd/src/phobos/std/datetime/timezone.d(546): called from
here: (*& singleton)()
/Library/D/dmd/src/phobos/std/datetime/systime.d(524): called from here:
opCall()
/Library/D/dmd/src/phobos/std/datetime/systime.d(472): called from here:
this.this(dateTime, zero(), tz)
test.d(6): called from here: SysTime(0L, Rebindable(null,
)).this(fromISOString("20220101T00"), null)
-
I'm sure there must be a better way to do this. But I couldn't find it
in the documentation.
It's trying to look up the local timezone at compile time.
You need to specify a time zone:
static time =
SysTime(DateTime.fromISOString("20220101T00"),
UTC()).toUnixTime;
-Steve
Re: How to get the element type of an array?
On 8/25/20 4:38 AM, Jon Degenhardt wrote: On Tuesday, 25 August 2020 at 05:02:46 UTC, Basile B. wrote: On Tuesday, 25 August 2020 at 03:41:06 UTC, Jon Degenhardt wrote: What's the best way to get the element type of an array at compile time? Something like std.range.ElementType except that works on any array type. There is std.traits.ForeachType, but it wasn't clear if that was the right thing. --Jon I'm curious to know what are the array types that were not accepted by ElementType ( or ElementEncodingType ) ? Interesting. I need to test static arrays. In fact 'ElementType' does work with static arrays. Which is likely what you expected. Also note that due to autodecoding, ElementType says `dchar` for strings. ElementEncodingType should be the choice if you are looking for the array element type. But you could also use the techniques specified here (and might be less confusing). But, if std.range is imported, a static array does indeed get a 'front' member. It doesn't satisfy isInputRange, but it does have a 'front' element. Because you can't pop the front of a static array. front works because a static array automatically casts to a normal array (there is no specialized overload for static arrays). The situation is still confusing though. If only 'std.range.ElementType' is imported, a static array does not have a 'front' member, but ElementType still gets the correct type. (This is where the documentation says it'll return void.) You are maybe thinking of how C works? D imports are different, the code is defined the same no matter how it is imported. *your* module cannot see std.range.primitives.front, but the range module itself can see that UFCS function. This is also why ElementType will fail on types that have UFCS front defined, but not imported directly from std.range.primitives. -Steve
Re: __FILE__ and __LINE__ in case of import expression
On 8/23/20 8:42 AM, Andrey Zherikov wrote:
On Saturday, 22 August 2020 at 03:43:10 UTC, Steven Schveighoffer wrote:
On 8/21/20 6:34 PM, Adam D. Ruppe wrote:
On Friday, 21 August 2020 at 22:12:48 UTC, Steven Schveighoffer wrote:
And honestly, if it says the source is "mixin-50, line 1", I think
people will get it.
I could probably live with that too, but the status quo is pretty
useful as-is.
I wonder if the compiler could detect when you are using a string
literal vs. a generated or imported string, and change the behavior
accordingly.
As far as I understand behavior of this is that mixin() changes __FILE__
to point to the location of this mixin. But import expression doesn't do
so. So if import("file") could change __FILE__ to "file" and __LINE__ to
1 internally that will make sense IMHO.
I mean something like this:
//__FILE__='test', __LINE__=1
mixin( //__FILE__='test-mixin-2', __LINE__=2
//__FILE__='test-mixin-2', __LINE__=3
import("file") //__FILE__='file', __LINE__=1 - only inside
import() !!!
//__FILE__='test-mixin-2', __LINE__=5
)
//__FILE__='test', __LINE__=7
import("file") returns a string, the source of the string is no longer
available for the compiler after it's done importing it as a string.
However, you can easily use the #line directive to automate this...
string getImport(string file)()
{
return "#line 1 " ~ file ~ "\n" ~ import(file);
}
mixin(getImport!"file");
-Steve
Re: __FILE__ and __LINE__ in case of import expression
On 8/21/20 6:34 PM, Adam D. Ruppe wrote: On Friday, 21 August 2020 at 22:12:48 UTC, Steven Schveighoffer wrote: And honestly, if it says the source is "mixin-50, line 1", I think people will get it. I could probably live with that too, but the status quo is pretty useful as-is. I wonder if the compiler could detect when you are using a string literal vs. a generated or imported string, and change the behavior accordingly. -Steve
Re: __FILE__ and __LINE__ in case of import expression
On 8/21/20 5:56 PM, Adam D. Ruppe wrote:
On Friday, 21 August 2020 at 21:42:21 UTC, Steven Schveighoffer wrote:
While not necessarily a "bug", it's not very useful.
Maybe not in this case, but it is perfectly accurate for cases like:
mixin(q{
some code here
});
Where it will actually line back up to the original file's line number
perfectly.
Who does that though? Why not just write "some code here" right in the
file? (I know that the string interpolation DIP might make this more likely)
And honestly, if it says the source is "mixin-50, line 1", I think
people will get it.
Whereas, if the code is like:
mixin(generateTheMixin());
and it says the line is 67, and line 67 has nothing to do with the mixin
source or the location it's mixed in, but instead, you need to subtract
the line number of mixing in from 67 to get the *real* line number, I
think the utility is really gone at that point.
-Steve
Re: __FILE__ and __LINE__ in case of import expression
On 8/21/20 5:08 PM, Adam D. Ruppe wrote: On Friday, 21 August 2020 at 21:06:11 UTC, Steven Schveighoffer wrote: The hybrid line number (original source line number + mixin line number) seems like a bug to me. I'm not so sure without seeing all the code. Remember to the compiler, the mixin thing is just a big string literal at the location of the import statement. So it adds the number of \n's in the string literal to the original line number to get the mixin line number. Look at the OP. It says line 22. Neither test.d nor the imported foo.d has 22 lines. While not necessarily a "bug", it's not very useful. The compiler should output a useful line number. I shouldn't have to do math to figure out what it "really" means. I'd argue that should be the line number based on the mixin source. The file name already is based on the line the string was mixed in, so both items contain useful data. To be fair to the compiler, when you are mixing in an import, the mixin doesn't know that the string came from an imported file, so it can't really determine the file automatically. But the line number should be reasonable. -Steve
Re: __FILE__ and __LINE__ in case of import expression
On 8/21/20 4:54 PM, Andrey Zherikov wrote:
On Friday, 21 August 2020 at 20:44:27 UTC, Andrey Zherikov wrote:
Thanks for this link! I can use "#line" to fix line number but not
file name:
file: 'foo.d-mixin-1', line: '6', module: 'test',
function: 'test.main', pretty function: 'int test.main(string[] args)',
file full path: 'C:\Users\andrey\foo.d-mixin-1'
I can actually fix this issue as well.
Changes in test.d:
test(); // line #16 (1)
mixin("#line 1 \"foo.d\"\n" ~ import("foo.d")); // line #17 (2)
test(); // line #18 (3)
Output:
file: 'test.d', line: '16', module: 'test',
function: 'test.main', pretty function: 'int test.main(string[] args)',
file full path: 'C:\Users\andrey\test.d'
file: 'foo.d', line: '6', module: 'test',
function: 'test.main', pretty function: 'int test.main(string[] args)',
file full path: 'C:\Users\andrey\foo.d'
file: 'test.d', line: '18', module: 'test',
function: 'test.main', pretty function: 'int test.main(string[] args)',
file full path: 'C:\Users\andrey\test.d'
Was just in the process of responding with this technique!
I think what you probably did first is:
int main(string[] args)
{
test();
#line 1 "foo.d"
mixin(import("foo.d"));
return 0;
}
Which sets the line and file of test.d at that point. But when the mixin
happens, I believe the parser/lexer sets the filename, but does not set
the line number to something different.
The hybrid line number (original source line number + mixin line number)
seems like a bug to me.
-Steve
Re: __FILE__ and __LINE__ in case of import expression
On 8/21/20 10:01 AM, Andrey Zherikov wrote: How can I get __FILE__ and __LINE__ values correct in case of import expression? ... So the output from line #16 (1) is correct although from line #17 (2) is not: file name is neither 'test.d' not 'foo.d' and line number is 22 although both test.d and foo.d are shorter. You can override the filename and line number to the lexer: https://dlang.org/spec/lex.html#special-token-sequence vibe.d does this so when errors from the trans-piled diet files happen, they match (mostly) back to the diet file, not the source file where they are mixed in. I understand that I can create a workaround but want to check first whether this is desired behavior or a bug that should be fixed? That's a good question. I would say it should say line 17 or line 6 (preferably the latter). That may be considered a bug, I don't know. -Steve
Re: SIGUSR1 in clock_nanosleep()? how to isolate this issue?
On 8/20/20 2:58 PM, mw wrote: Hi, I run into an issue: it's SIGUSR1 in clock_nanosleep() The GC uses SIGUSR1 (and SIGUSR2). Maybe that's the issue? https://github.com/dlang/druntime/blob/e1fb19829ebef0419782de43ce4b0e2a1ba140be/src/core/thread/osthread.d#L1946 -Steve
Re: how stdin stream works?
On 8/19/20 1:44 PM, Flade wrote:
Hi everyone! I'm trying to do error handling (with the try block) and
when I give a wrong value to the variable (it is an integer and I give a
non-number value), then It doesn't let me re get input. The code:
int x;
bool not_accepted = false;
while (!not_accepted) {
try {
write("x: ");
readf("%d\n", x);
not_accepted = true;
} catch (Exception msg) {
writeln("Please give a right coordinate");
}
Probably readf stops as soon as it encounters an error, meaning that
your input is still present.
Try instead getting a line via readln, and then trying to read that into
your expected input.
-Steve
Re: `enum x;` - what is it?
On 8/19/20 10:06 AM, Victor Porton wrote:
This declaration does compile:
enum x;
But what is it? Is it an equivalent of
enum x { }
?
What in the specification allows this looking a nonsense
enum x;
?
I use it as a symbol for UDAs.
enum required;
struct S
{
@required int x;
}
which can then easily be found.
What is it? I have no idea. I'm just using the name.
I think it's treated as a forward declaration. Kind of like
void foo();
-Steve
Re: Types of lambda args
On 8/16/20 8:27 PM, Adam D. Ruppe wrote: On Monday, 17 August 2020 at 00:20:24 UTC, Cecil Ward wrote: In a lambda, how do we know what types the arguments are? In something like (x) => x * x In that the compiler figures it out from usage context. So if you pass it to a int delegate(int), it will figure x must be int. It's actually a template, with some special benefits. It relies on IFTI to work. -Steve
Re: Autodecode?
On 8/16/20 4:53 PM, JN wrote: Related to this thread: https://forum.dlang.org/post/xtjzhkvszdiwvrmry...@forum.dlang.org I don't want to hijack it with my newbie questions. What is autodecode and why is it such a big deal? From what I've seen it's related to handling Unicode characters? And D has the wrong defaults? Aside from what others have said, autodecode isn't really terrible as a default. But what IS terrible is the inconsistency. Phobos says char[] is not an array, but the language does. e.g.: char[] example; static assert(!hasLength!(typeof(example))); // Phobos: no length here! auto l = example.length; // dlang: um... yes, there is. static assert(!hasIndexing!(typeof(example))); // P: no index support! auto e = example[0]; // D: yes, you can index. And probably my favorite WTF: static assert(is(ElementType!(typeof(example)) == dchar)); // P: char is a range of dchar! foreach(e; example) static assert(is(typeof(e)) == char)); // D: nope, it's an array of char This leads to all kinds of fun stuff. Like try chaining together several char[] arrays, and then converting the result into an array. Surprise! it's a dchar[], and you just wasted a bunch of CPU cycles decoding them, not to mention the RAM to store it. But then Phobos, as it's telling you that all these things aren't true, then goes behind your back and implements all kinds of special cases to deal with these narrow strings *using the array interfaces* because it performs better. We will be much better off to be done with autodecoding. And for many cases, autodecoding is just fine! Most of the time, you only care about the entire string and not what it's made of. Many other languages do "autodecoding", and in fact the string type is opaque. But then it gives you ways to use it that aren't silly (like concatenating 2 strings knows what the underlying types are and figures out the most efficient way possible). If `string` was a custom type and not an array, we probably wouldn't have so many issues with it. -Steve
Re: Cannot call @system funciton (stdout)
On 8/16/20 6:07 AM, Simen Kjærås wrote:
On Saturday, 15 August 2020 at 23:59:36 UTC, Joel wrote:
../../JMiscLib/source/jmisc/base.d(176,2): Error: @safe function
jmisc.base.upDateStatus!string.upDateStatus cannot call @system
function std.stdio.makeGlobal!"core.stdc.stdio.stdout".makeGlobal
/Library/D/dmd/src/phobos/std/stdio.d(4837,20):
std.stdio.makeGlobal!"core.stdc.stdio.stdout".makeGlobal is declared here
I got around it by avoiding 'stdout'.
First, what's wrong with using writeln and friends instead of directly
mucking about with stdout? :p
stdout is __gshared, so it's available on any thread at any time. That's
not @safe, so it's @system.
If you know you're not using stdout from multiple threads, or don't care
(it might be perfectly safe even though it's possible to misuse), you
can use this code:
@property File trustedStdout() @trusted
{
return stdout;
}
That's a @trusted wrapper that you can call from @safe code. It's not
actually safe though, as multiple threads could be using trustedStdout
at the same time. In many use cases, this is unlikely to matter, but
it's wroth keeping in mind.
Technically, there's nothing unsafe about reading stdout, as long as you
are not setting it. Otherwise, writeln wouldn't be @safe (as all it does
is call a private trustedStdout).
The whole thing is messy IMO, and should be revisited.
-Steve
Re: Leaving a pointer to it on the stack
On 8/13/20 4:51 PM, Andre Pany wrote: On Thursday, 13 August 2020 at 20:11:50 UTC, Steven Schveighoffer wrote: The garbage collector scans all of the stack as if it were an array of pointers. So if you have a pointer to it anywhere on the stack, it won't be collected. However, it only scans threads that the runtime knows about. If I understand it right, this paragraph doesn't help in my real scenario where sample (Dll) is called from a delphi executable. So in your real world scenario, a non-D thread/program is calling sample, and it controls the location of *i? If so, then no, you can't depend on D not collecting that data, because D might not scan that location. It is another runtime (delphi). But on the other side, as D GC only runs if something new should be allocated on D side, I can safely assume that the Delphi caller can access the heap variables? Of course as long as it doesn't store the references and use it later... As long as you aren't allocating again later, yes. You can also disable collections and only run them when you know it's safe to do so. -steve
Re: Leaving a pointer to it on the stack
On 8/13/20 4:04 PM, Andre Pany wrote:
Hi,
in the specification
https://dlang.org/spec/interfaceToC.html#storage_allocation there is
this paragraph:
"Leaving a pointer to it on the stack (as a parameter or automatic
variable), as the garbage collector will scan the stack."
I have some trouble to understand what does this mean. Given this example:
```
import std;
void main()
{
int* i;
sample();
writeln(*i);
}
extern(C) export void sample(int** i)
{
*i = new int();
**i = 42;
}
```
Int variable is created on the heap. How do I leave a pointer on the stack?
(In the real coding, sample function will be called from Delphi)
The garbage collector scans all of the stack as if it were an array of
pointers. So if you have a pointer to it anywhere on the stack, it won't
be collected.
However, it only scans threads that the runtime knows about.
-Steve
Re: vibe.d and my first web service
On 8/13/20 3:28 AM, WebFreak001 wrote: On Wednesday, 12 August 2020 at 21:11:54 UTC, Daniel Kozak wrote: [...] Unfortunately, I think vibe-d is dead. With every release it is worse than before and it seems there is almost no activity. So D really need new champion here maybe hunt will be next champion. Can you give an example how vibe.d gets worse? It's really stable right now and while I would enjoy the feature PRs to go in more quickly, I think they are pretty well reviewed when they get reviewed and eventually once merged it's only good code that makes it in. The environment just updated and made previous versions more unstable, most of which is inherited into current versions too, but these things are getting fixed. Like for example a lot of linux distros changed to OpenSSL 1.1 which broke HTTPS client calls in vibe.d and was fixed in 0.8.6 or MongoDB 3.4+ changed a lot of things like indexes which were then broken which was fixed in 0.9.0 now though. I haven't had a really big problem with vibe.d, except for the ctrl-c bug (which I've worked around as described). I agree with the OP of this subthread that it is the most important problem for vibe.d (and vibed-core really). I wish I knew how to fix it... My experience with getting features into vibe.d has been good, I've added a few and Sonke has been very receptive, even if his responses are delayed (he must be really busy). I've been there too. -Steve
Re: DMD: how to restore old unittest+main
On 8/13/20 5:02 AM, Nils Lankila wrote: On Thursday, 13 August 2020 at 08:49:21 UTC, WebFreak001 wrote: On Thursday, 13 August 2020 at 07:52:07 UTC, novice3 wrote: Hello. I don't use dub. I use Windows and *.d file association to compile small apps by dmd with "-i -unittest -g" switches. Now i update dmd, and found, that apps compiled with "-unittest" not runs main(). How i can restore old behaviour (run all unittests then main()) *without use "--DRT-testmode=run-main" switch every time then i start compiled app.exe*? I want just press Enter on app.d file, then press Enter on app.exe. Any advises? Thanks. Try version (unittest) extern(C) __gshared string[] rt_options = [ "testmode=run-main" ]; Yeah that works but we should really have a way to do this programmatically, in a way it is already, but by calling function, not by the bias of a string that get parsed. https://dlang.org/phobos/core_runtime.html#.Runtime.extendedModuleUnitTester Though I highly recommend using the rt_options mechanism if all you are after is the original behavior, it's much simpler. -Steve
Re: Reading from stdin significantly slower than reading file directly?
On 8/12/20 6:44 PM, methonash wrote: Hi, Relative beginner to D-lang here, and I'm very confused by the apparent performance disparity I've noticed between programs that do the following: 1) cat some-large-file | D-program-reading-stdin-byLine() 2) D-program-directly-reading-file-byLine() using File() struct The D-lang difference I've noticed from options (1) and (2) is somewhere in the range of 80% wall time taken (7.5s vs 4.1s), which seems pretty extreme. For comparison, I attempted the same using Perl with the same large file, and I only noticed a 25% difference (10s vs 8s) in performance, which I imagine to be partially attributable to the overhead incurred by using a pipe and its buffer. So, is this difference in D-lang performance typical? Is this expected behavior? Was wondering if this may have anything to do with the library definition for std.stdio.stdin (https://dlang.org/library/std/stdio/stdin.html)? Does global file-locking significantly affect read-performance? For reference: I'm trying to build a single-threaded application; my present use-case cannot benefit from parallelism, because its ultimate purpose is to serve as a single-threaded downstream filter from an upstream application consuming (n-1) system threads. Are we missing the obvious here? cat needs to read from disk, write the results into a pipe buffer, then context-switch into your D program, then the D program reads from the pipe buffer. Whereas, reading from a file just needs to read from the file. The difference does seem a bit extreme, so maybe there is another more complex explanation. But for sure, reading from stdin doesn't do anything different than reading from a file if you are using the File struct. A more appropriate test might be using the shell to feed the file into the D program: dprogram < FILE Which means the same code runs for both tests. -Steve
Re: __vector(ubyte[32]) misalignment
On 8/10/20 2:53 PM, Bruce Carneal wrote: No biggee but it looks like there is some duplicate code at the end of the __alignPad unittest. Hah! I think I copy-pasted that intending to write a new test, but then tried it separately and found another issue (typeid(__vector(ubyte[32])).talign returned 16!) So I forgot to go back in and delete that case. Thanks -Steve
Re: __vector(ubyte[32]) misalignment
On 8/9/20 8:46 AM, Steven Schveighoffer wrote: On 8/9/20 8:37 AM, Steven Schveighoffer wrote: I think this has come up before, there may even be a bug report on it. Found one, I'll see if I can fix the array runtime: https://issues.dlang.org/show_bug.cgi?id=10826 Bruce, I have a PR to hopefully fix these issues, if you want to test against it: https://github.com/dlang/druntime/pull/3192 -Steve
Re: Factory pattern for classes
On 8/9/20 10:27 AM, lexxn wrote:
On Sunday, 9 August 2020 at 12:24:05 UTC, Steven Schveighoffer wrote:
Object getClassById(uint id)
{
if (id == 0) {
return new A;
} else if(id == 1) {
return new B;
} else {
return new C;
}
}
I assume that the correct syntax for the getClassById is
Object getClassById(uint id) {
if (id == 0) {
return new A();
} else if (id == 1) {
return new B();
} else {
return new C();
}
}
or maybe I'm wrong.
If the constructor has no parameters, classes can be new'd without
parentheses, so your modification results in identical code as my example.
-Steve
Re: Factory pattern for classes
On 8/9/20 10:58 AM, lexxn wrote: On Sunday, 9 August 2020 at 12:24:05 UTC, Steven Schveighoffer wrote: If you know what your class is going to be, I'd just import the file that contains it and avoid the whole Object.factory deal. It's going to go away anyways. Not sure if it's a good idea in my case. I'm going to be using all of the classes eventually in my loop, but I just need to initialize them in a conditional way depending on the use case. My point is -- it's a good idea to not use Object.factory, because it's going to go away. It doesn't work in all cases anyway. -Steve
Re: __vector(ubyte[32]) misalignment
On 8/9/20 8:37 AM, Steven Schveighoffer wrote: I think this has come up before, there may even be a bug report on it. Found one, I'll see if I can fix the array runtime: https://issues.dlang.org/show_bug.cgi?id=10826 -Steve
Re: __vector(ubyte[32]) misalignment
On 8/9/20 8:09 AM, Bruce Carneal wrote:
On Sunday, 9 August 2020 at 09:58:18 UTC, Johan wrote:
On Sunday, 9 August 2020 at 01:03:51 UTC, Bruce Carneal wrote:
The .alignof attribute of __vector(ubyte[32]) is 32 but initializing
an array of such vectors via an assignment to .length has given me 16
byte alignment (and subsequent seg faults which I suspect are related).
Is sub .alignof alignment expected here? IOW, do I have to manually
manage memory if I want alignments above 16?
Do you have a code example?
And what compiler are you using?
-Johan
At run.dlang.io recent runs of both dmd and lcd compilations of the
below revealed misalignment.
import std;
void main() @safe
{
alias V = __vector(ubyte[32]); // requires -mcpu=native or other on
cmd line
V[] va;
size_t misalignments;
foreach(N; 1..101) {
va.length = N;
const uptr = cast(ulong)va.ptr;
misalignments += (uptr % V.alignof) != 0;
}
writefln("misaligned %s per cent of the time", misalignments);
}
All blocks in the GC that are more than 16 bytes are aligned by 32
bytes. You shouldn't have any 16 byte blocks here, because each element
is 32 bytes long.
However, if your block grows to a page size, the alignment will be 16
bytes off (due to the metadata stored at the front of the block).
A page size is 4096 bytes. So anything larger than 2048 will require a
page-sized block or larger.
I would guess that once your array gets longer than 63 elements,
it's always misaligned?
The current code ensures a 16 byte alignment. That really should go to
32 (for this reason). I think this has come up before, there may even be
a bug report on it.
See:
https://github.com/dlang/druntime/blob/660d911bbd3342c1f1c1478d12e3e943c6038da0/src/rt/lifetime.d#L35
The other thing you can do is avoid allocating using the array runtime,
and just allocate using the GC calls directly. This means appending
won't work, and neither will destructors (though that shouldn't be
important here).
Question for those in the know: are there any other alignments that we
should ensure are possible?
-Steve
Re: String argument with optional value in std.getopt
On 8/8/20 7:58 PM, Hassan wrote:
Hello
I'm trying to get getopt to recognize an argument that may or may not
take a value. Here's an example :
../hashtrack --list
../hashtrack --list filter
The problem is that if I point list to a string variable, the first call
fails with "Missing value for argument --list".
I tried having it point to a callback with an optional second parameter :
void cb(string option, string value = "")
{
writeln("--list");
option.writeln();
value.writeln();
}
But I get the same error. I also tried having two callbacks with the
same name. The first one takes a single argument, and the second one
accepts two :
void cb(string option)
{
writeln("--list");
option.writeln();
}
void cb(string option, string value)
{
writeln("--list");
option.writeln();
value.writeln();
}
But it only calls the first one. I'm using DMD64 D Compiler v2.090.0.
Any pointers ?
getopt doesn't support optional parameters.
Two things I can think of:
1. Have 2 options that do the same thing, but only one accepts a
parameter (e.g. `--list` and `--listf filter`)
2. If your optional parameters are not tied to the option itself, then
don't accept them via getopt. In other words, if `hashtrack filter` is
supposed to be valid, then filter isn't an option after all, it's a
standard parameter.
-Steve
Re: Factory pattern for classes
On 8/9/20 5:16 AM, lexxn wrote:
I'm trying to get the factory pattern going with classes
class A {}
class B {}
class C {}
auto getClassById(uint id)
{
if (id == 0) {
return cast(A)Object.factory("A");
} else if(id == 1) {
return cast(B)Object.factory("B");
} else {
return cast(C)Object.factory("C");
}
}
I'm getting 2 errors:
main.d(69): Error: Expected return type of main.A, not main.B:
main.d(67): Return type of main.Ainferred here.
Change your return type to Object.
Also is it possible to completely skip the getClassById function and if
I've and array string classes = ["A", "B"] to just
cast(classes[0])Object.factory(classes[0]);. I understand that I need to
cast classes[0].
This won't work.
If you know what your class is going to be, I'd just import the file
that contains it and avoid the whole Object.factory deal. It's going to
go away anyways.
In other words:
Object getClassById(uint id)
{
if (id == 0) {
return new A;
} else if(id == 1) {
return new B;
} else {
return new C;
}
}
-Steve
Re: problem using std.format on Arm
On 8/8/20 1:00 PM, Jeremiah Glover wrote: What can I do to fix this and get the most recent version of phobos that will compile? You are using an old version of the compiler. formattedRead used to accept only pointers, not by ref. See https://github.com/dlang/phobos/pull/5009 Try doing: formattedRead(line, "%d\t%s", , ); Figure out which version of the compiler you have, and then use the docs for that version by downloading the compiler and using the local html documentation. Alternatively, see if you can find a newer compiler that suports ARM. -Steve
Re: iopipe code to count lines in gzipped file works with v 0.1.7 but fails with 0.2.1
On 8/7/20 9:31 PM, Steven Schveighoffer wrote: On 8/7/20 8:57 PM, Steven Schveighoffer wrote: I think this is an issue with dub when using an inline recipe file, but I don't know? ugh. This is an issue with iopipe specifying io version 0.2.x. I will fix this. OK, iopipe 0.2.2 is released, with no specific io dependency. It should work with io 0.3.1 as a separate listed dependency. -Steve
Re: iopipe code to count lines in gzipped file works with v 0.1.7 but fails with 0.2.1
On 8/7/20 8:57 PM, Steven Schveighoffer wrote: I tried adding dependency "io" version="~>0.3.0" But it fails with: Got no configuration for dependency io ~>0.3.1 of hello ~master!? If I add dependency "io" version="*" it works. I think this is an issue with dub when using an inline recipe file, but I don't know? ugh. This is an issue with iopipe specifying io version 0.2.x. I will fix this. -Steve
Re: iopipe code to count lines in gzipped file works with v 0.1.7 but fails with 0.2.1
On 8/7/20 9:40 AM, Andrew wrote: Hi, This code to count lines in a gzipped file exits with "Program exited with code -9" when run with the latest version of the library, I guess because I am doing unsafe things. BTW the safety improvements only change whether it compiles as @safe or not. If it's building but running is exiting with a code then it's possible there's a bug somewhere. I did have to change a lot of code to get it to build properly. If you have it building, but it's exiting with an error, then please file an issue with an example file and sample source that causes the issue. -Steve
Re: iopipe code to count lines in gzipped file works with v 0.1.7 but fails with 0.2.1
On 8/7/20 9:40 AM, Andrew wrote:
Hi,
This code to count lines in a gzipped file exits with "Program exited
with code -9" when run with the latest version of the library, I guess
because I am doing unsafe things. Could someone tell me how to change it
to make it work? The actual program I'm writing processes a file line by
line, so ideally I'd like to keep the structure of open a file, then
foreach over it.
Thanks very much
As of iopipe v0.2.0, io is no longer a required dependency, it's
optional. So you must also add a dependency for io.
I tried adding
dependency "io" version="~>0.3.0"
But it fails with:
Got no configuration for dependency io ~>0.3.1 of hello ~master!?
If I add
dependency "io" version="*"
it works.
I think this is an issue with dub when using an inline recipe file, but
I don't know?
Note that in this simple example, the line count is stored in the line
pipe, you can retreive the number of lines by accessing the `segments`
member of the pipe (undocumented, I have to fix that). So my code looks
like:
---
/+ dub.sdl:
name "hello"
dependency "iopipe" version="~>0.2.0"
dependency "io" version="*"
+/
import std.stdio;
//import std.typecons; // refCounted not @safe
import iopipe.textpipe;
import iopipe.zip;
import iopipe.bufpipe;
import iopipe.refc; // refCounted that is @safe
import std.io : File = File; // just a note, I don't know why you are
renaming here...
void main() @safe // yay @safe!
{
auto counter = 0;
auto fileToRead = File("file.gz").refCounted.bufd
.unzip(CompressionFormat.gzip)
.assumeText
.byLine;
fileToRead.process();
writeln(fileToRead.segments);
}
---
FYI, I noticed that in my simple test, this outputs one less than the
actual lines. I'll have to look into *that* too.
That dependency on writeln also irks me ;) I need to get working on that
iopipe replacement for it...
-Steve
Re: Is there an alternative of `go get`
On 8/7/20 3:28 AM, Cogitri wrote: On Friday, 7 August 2020 at 07:17:25 UTC, Ky-Anh Huynh wrote: Hi everyone, `go get` in Golang world has a simple way to fetch and install binary ``` $ go get github/foo/bar.git $ export PATH=$PATH:$(go env GOPATH)/bin $ bar --help ``` This saves a lot of time and setup. Is that an alternative when using dub? Hello, you can use `dub build $name` to build the package. The resulting binary should be in $HOME/.dub/$name-$version/$name/bin then. You can also run the binary via `dub run $name -- $additional_args`, then you won't have to add that path to your PATH. If you haven't run `dub build $name` previously, `dub run $name` will do that for you. Having a way to configure the path would be really helpful. I'll add a request for dub. -Steve
Re: Non-recursive maxSizeOf
On 8/6/20 9:23 AM, Adam D. Ruppe wrote:
On Thursday, 6 August 2020 at 13:18:40 UTC, Per Nordlöw wrote:
mixin(T.stringof ~ " _store" ~ T.mangleof ~
Never ever use mixin(T.stringof). Always just use mixin("T") instead.
mixin("T _store", T.mangleof /* or just idx is gonna be better */,";");
Though I doubt this is going to win a benchmark anyway getting this
complicated.
Well, one *could* do:
static if(is(union { Ts t; }))
and use the union for "normal" cases.
I think the CTFE solution is cleaner. Is there a good reason to avoid it?
-Steve
Re: std.conv.ConvException from double to uint64_t, but only locally in a large project
On 8/4/20 2:15 PM, Steven Schveighoffer wrote: I'll file a bug. https://issues.dlang.org/show_bug.cgi?id=21112 -Steve
Re: std.conv.ConvException from double to uint64_t, but only locally in a large project
On 8/4/20 1:36 PM, drathier wrote:
I'm getting a crash when I'm converting a double to an uint64_t.
```
std.conv.ConvException@/usr/local/opt/dmd/include/dlang/dmd/std/conv.d(2054):
Value (1596) does not match any member value of enum '__c_ulonglong'
```
I've narrowed down the code to this:
```
static import std.conv;
double thing = 42.0;
std.conv.to!(uint64_t)(thing);
```
which works just fine on https://run.dlang.io/ or in a single-file dmd
invocation. When I compile and run it locally on my mac as part of a
specific large app using dub, it always crashes like this.
Where would I even start debugging this?
DMD64 D Compiler v2.093.0
DUB version 1.22.0, built on Jul 9 2020
So a common way to typedef something is to use enum:
enum uint64_t : ulong;
This gives you a new type that works pretty much just like ulong, but
will not implicitly convert from ulong.
However, std.conv.to is likely interpreting this as an enumeration type,
where it has to match one of the enum members. But since this isn't an
enumeration in the standard way, it fails (it has no members!)
This code reproduces the problem on run.dlang.io:
void main()
{
import core.stdc.config;
import std.conv;
auto x = 0.5;
auto y = x.to!__c_ulonglong;
}
Note the code in druntime which defines __c_ulonglong:
https://github.com/dlang/druntime/blob/0db2e65bba7cc319309bd32957763882870d5b03/src/core/stdc/config.d#L121
I'll file a bug.
-Steve
Re: Template functions inside interface
On 8/4/20 9:39 AM, Adam D. Ruppe wrote:
On Tuesday, 4 August 2020 at 13:36:15 UTC, Zans wrote:
Is there any way to declare template functions inside interface and
then override them in a class?
No, the templates in the interface are automatically considered `final`.
So the body must be in the interface too to avoid that undefined
reference error.
You can have them forward to normal methods in the interface though,
just there needs to be a fixed number of them with concrete types.
I was kind of surprised the compiler didn't complain about override
there. Is override ever a valid attribute for a template function? Is
there a reason we ignore it for templates?
I can imagine a lot of confusion for something like:
import std.stdio;
interface MyInterface
{
T doAndReturnSomething(T)(T param){return T.init;}
}
class MyClass : MyInterface
{
override T doAndReturnSomething(T)(T param)
{
return param;
}
}
void main()
{
MyInterface myClass = new MyClass();
writeln(myClass.doAndReturnSomething("Hello"));
}
Which compiles, runs, and prints nothing.
-Steve
Re: safety and auto vectorization
On 8/3/20 3:22 PM, Bruce Carneal wrote: Thanks Steve (and Chad). Summary: underspecified, varying behavior across versions, buggy. Steve, what's the best way for me to report this? Are spec issues lumped in with the other bugzilla reports? Yep. You can file under dlang.org with the spec keyword. Although this looks like it's not exactly a spec issue. I'd start it as a dmd bug (I don't know the exact interaction with the compiler and the library). It might also be a druntime bug. -Steve
Re: safety and auto vectorization
On 8/2/20 1:31 PM, Bruce Carneal wrote:
import std;
void f0(int[] a, int[] b, int[] dst) @safe {
dst[] = a[] + b[];
}
void f1(int[] a, int[] b, int[] dst) @trusted {
const minLen = min(a.length, b.length, dst.length);
dst[0..minLen] = a[0..minLen] + b[0..minLen];
assert(dst.length == minLen);
}
I was surprised that f0 ran just fine with a.length and b.length geq
dst.length. Is that a bug or a feature?
Assuming it's a feature, are f0 and f1 morally equivalent? I ask
because f1 auto-vectorizes in ldc while f0 does not. Not sure why. As
a guess I'd say that the front end doesn't hoist bounds checks in f0 or
at least doesn't convey the info to the back end in a comprehensible
fashion. Non-guesses welcome.
First, I think this is a bug. A regression in fact. As of 2.077 this
works, and before it did not. There is nothing in the spec that says the
behavior is defined for this case.
Second, it's more than just that. This also runs currently:
void main()
{
auto a = [1, 2, 3];
auto b = [4, 5, 6];
int[] dst = new int[4]; // note the extra element
dst[] = a[] + b[];
writeln(dst[3]);
}
Prior to 2.077, this fails with array length problems.
After that it prints (at the moment): 402653184
If I up the size to 5, it fails with a range violation. I strongly
suspect some off-by-one errors, but this looks unsafe.
-Steve
Re: Idiomatic D code to avoid or detect devision by zero
On 8/3/20 5:53 AM, Martin Tschierschke wrote: On Friday, 31 July 2020 at 14:18:15 UTC, Steven Schveighoffer wrote: On 7/31/20 9:55 AM, Martin Tschierschke wrote: What would be the idiomatic way to write a floating point division occuring inside a loop and handle the case of division by zero. c = a/b; // b might be zero sometimes, than set c to an other value (d). (In the moment I check the divisor being zero or not, with an if-than-else structure, but I find it ugly and so I ask here.) c = b == 0 ? d : a/b; I don't think a function would be shorter or clearer... c = div(a, b, d); Alternatively, you could use a type to effect the behavior you want. Thanks, for the hints. I find the ? : - expressions sometimes hard to reed, especially when a and b are not so simple expressions. I prefer putting additional bracket around: c = (b_expression == 0) ? (d_longer_expression) : (a_expression/b_expression); Yes, that is fine, and up to your preference. You may actually need the parentheses if the expressions somehow override the precedence of the ?: operator. Even with symbol uses, I personally would do actually: c = (b == 0 ? d : a/b); Just because the ` = b == ` looks really bad to me. -Steve
Re: dynamic array .length vs .reserve - what's the difference?
On 7/31/20 12:32 PM, wjoe wrote: On Friday, 31 July 2020 at 04:28:57 UTC, Ali Çehreli wrote: Another option, which is curiously said to be more performant in memory allocation than native arrays, is std.array.Appender. I've used function-local static Appenders to cut down on memory allocation. Here is an uncompiled pseudo code: [...] This looks like an even better way to do it. Thanks, Ali :) Just FYI, the reason this is faster is because there is no need to go through the opaque calls into druntime to figure out if appending-in-place is possible. The reserved length is stored directly in the struct. -Steve
Re: Idiomatic D code to avoid or detect devision by zero
On 7/31/20 9:55 AM, Martin Tschierschke wrote: What would be the idiomatic way to write a floating point division occuring inside a loop and handle the case of division by zero. c = a/b; // b might be zero sometimes, than set c to an other value (d). (In the moment I check the divisor being zero or not, with an if-than-else structure, but I find it ugly and so I ask here.) c = b == 0 ? d : a/b; I don't think a function would be shorter or clearer... c = div(a, b, d); Alternatively, you could use a type to effect the behavior you want. -Steve
Re: dynamic array .length vs .reserve - what's the difference?
On 7/30/20 11:58 AM, wjoe wrote:
I just stumbled upon code like this:
struct Foo(T)
{
T[] b;
this(int n)
{
b.reserve(n);
b.length = n;
}
}
..reserve looks redundant.
It is, in this case. Reserve will extend the allocated length to n, but
not adjust the slice, and setting the length will adjust the slice and
consume that data from the block. Just setting length has the same effect.
The docs are explaining .length nicely, however lack any specifics about
reserve.
Changing the length of an array may relocate and copy. New items are
initialized with T.init - is that true for both length and reserve ?
reserve preallocates data for use in appending, but does not alter the
length of the specified array.
It's like saying, "I'm going to append enough elements to this array so
the total length is N." It's very much tied to appending.
Note that reserve may do nothing, if there is already at least N
elements available.
Also there's .capacity - is that equivalent to reserve ?
Capacity tells you how many total elements the current reserved space
can hold. If the array is not appendable, then this returns 0.
Another curiosity I noticed is that the docs say that the runtime tries
to resize in place, however:
b = b[0..$-1]; if length==1 seems to collect the memory at once because
if it's immediately followed by b.length = 1; or b ~= someT; b.ptr
points to a new address.
Why ?
Because you would be overwriting the data already in the array.
For example:
auto a = b;
b = b[0 .. $-1];
b ~= someT;
If that last line is done in-place, then it overwrites a[$-1].
If you know that it's OK to do this, then you should call
assumeSafeAppend on the array (which will adjust the "used" space down
to the current array).
I know that b=b[0..0]; is equivalent to b = null;
No, if slicing b to b[0 .. 0], b.ptr does not change. b = null sets the
pointer to null. In the former case, if you called assumeSafeAppend on
it, then it could append in-place at that point. With the null pointer,
it would reallocate.
-Steve
Re: dub build to generate different kinds of libs
On 7/28/20 5:50 PM, jeff thompson wrote: Hello Im brand new to D (loving it btw!) and have decided to build a largish windows project in the language. First question, is there a dub.json setup to have a dub build to generate multiple binaries in one call? Like a dll and a static lib. Seems like it would be easy maybe im missing something obvious? Like a configuration with multiple targetTypes. Yes, just do that (make multiple configurations) You just have to build them separately: dub build --config=library dub build --config=application -Steve
Re: Why are std.bitmanip.bitfields so big ?
On 7/28/20 5:46 AM, MoonlightSentinel wrote: On Tuesday, 28 July 2020 at 09:28:27 UTC, wjoe wrote: It was run on the doc page. I suppose the examples are wrapped in a unittest block? Indeed, see https://github.com/dlang/phobos/blob/cd2ba0d2c378a893ec0eaefc57b87d0770a1990c/std/bitmanip.d#L293-L314 It doesn't necessarily need to be in a unittest block (AFAIK, the example executer isn't actually running the unittests of phobos), but it does need to be inside a function, because you have executable code there. Either way, yes, it's run inside a function, so you need to apply static to the struct to avoid the frame pointer. I tested that, and it prints 1 instead of 16. -Steve
Re: Help with Ranges
On 7/27/20 1:10 PM, Charles wrote: Still, I'm confused since, as far as I know, map wraps its source, i.e. the array in this case, which is sortable. It seems to me the only reason I can't sort MapResult is because it doesn't have the proper interface. Let's talk about a concrete example, so you can see why: int[] arr = [21, 83, 45, 60]; auto m = arr.map!(a => a % 10); Sort is going to look at m as a random-access range of integers. But the integer you get from m[2] for instance is going to be 5. So you can compare e.g. m[2] and m[3] (5 and 0), but how do you *WRITE* back to the original array? All you have as an interface is non-writable 5 and 0, not the original array members 45 and 60. In other words, if you swapped 5 and 0, map can't do the inverse transform here (and even if it could, 0 and 5 map to millions of possible original values). What it seems like you are possibly interested in doing is to sort the original array based on a transformation. But in your original post you said "doSomething is some predicate that does a lot of processing on each element", so I assumed, e.g. something like this is not valid for your use case: arr.sort!((a, b) => doSomething(a) < doSomething(b)) as it would be very expensive assuming doSomething is expensive. You can use H.S. Teoh's suggestion and use schwartzSort (in fact, it does something almost exactly the same as what I wrote, except it sorts the original elements). However, for something like a % 10, I'd much rather just do it without allocating another array. So it really depends on what your requirements are, which you haven't fully identified. -Steve
Re: Why are std.bitmanip.bitfields so big ?
On 7/27/20 5:49 AM, wjoe wrote:
struct A
{
mixin(bitfields!(
bool, "flag1", 1,
bool, "flag2", 1,
uint, "", 6));
}
Is this inside a function? If so, put `static` on it.
What you are seeing is the 8-byte frame pointer that comes from inner
structs so you can access stack variables inside the struct.
-Steve
Re: Help with Ranges
On 7/26/20 3:10 AM, Charles wrote:
Suppose I have the following line of code where arr is an array,
doSomething is some predicate that does a lot of processing on each
element, sort must come after the mapping, and there are more operations
done to the range after sort:
arr.map!doSomething.sort. ...;
Sort fails to instantiate because the range it's receiving doesn't
support element swapping. This may and might be resolved by calling array:
arr.map!doSomething.array.sort. ...;
However, this might trigger an allocation, and there's still more to do!
Is there something I'm missing with regards to ranges that could help me
make the first line work without using array, or is it more of an issue
with my code's design?
That is what you need to do.
A map that returns an lvalue would be sortable, but you would be sorting
the processed elements, and probably not the original elements.
I have found this handy tool quite useful in my code where I need a
temporary array:
// creates a concrete range (std.container.array.Array range) out of the
// original range that is eagerly fetched, and then can be processed,
without
// allocating extra garbage on the heap.
auto concreteRange(Range)(Range r)
{
import std.range : ElementType;
import std.container.array : Array;
return Array!(ElementType!Range)(r)[];
}
Slicing an Array will keep the reference count correctly, and destroy
the memory automatically after you're done using it. So it's perfect for
temporary arrays in pipelining.
-Steve
Re: Returning range of inout objects from inout method
On 7/25/20 4:07 PM, Steven Schveighoffer wrote:
On 7/25/20 3:16 PM, FreeSlave wrote:
On Saturday, 25 July 2020 at 14:19:15 UTC, Steven Schveighoffer wrote:
The only way to do this without code duplication (but with generated
code duplication) is to template the byAction function on the type of
`this`:
auto byAction(this This)() { /* same implementation */ }
Note that this ONLY works if your base range type is an array. If you
have a custom range type, you need to parameterize that based on the
constness of `This`.
Thanks. I thought this template is useful only in inheritance.
Is constness of member function inferred automatically in this case?
In this case, yes. I don't know actually if this is documented.
It should be now.
https://github.com/dlang/dlang.org/pull/2835
-Steve
Re: Returning range of inout objects from inout method
On 7/25/20 3:16 PM, FreeSlave wrote:
On Saturday, 25 July 2020 at 14:19:15 UTC, Steven Schveighoffer wrote:
The only way to do this without code duplication (but with generated
code duplication) is to template the byAction function on the type of
`this`:
auto byAction(this This)() { /* same implementation */ }
Note that this ONLY works if your base range type is an array. If you
have a custom range type, you need to parameterize that based on the
constness of `This`.
Thanks. I thought this template is useful only in inheritance.
Is constness of member function inferred automatically in this case?
In this case, yes. I don't know actually if this is documented.
-Steve
Re: Returning range of inout objects from inout method
On 7/25/20 8:26 AM, FreeSlave wrote:
I want to be able to return a range of const objects from the const
object and a range mutable objects from the mutable object.
inout comes to mind, but not applicable in this case, because inout must
be applied to the return type as whole, which does not make much sense
for the range. Defining range of inout objects does not work too. See
the example.
import std.range;
class A
{
string name;
}
class B
{
A[] _actions;
ForwardRange!(inout(A)) byAction() inout { // can't instance this type
import std.algorithm : filter;
return inputRangeObject(_actions.filter!(a => a !is null &&
a.name.length));
}
}
So how can I achieve the intended effect without code duplication? (like
making separate mutable and const versions of the method).
You can't exactly. The limitation that inout cannot be applied to a
struct member is a crappy limitation, one which I wish I never advocated
for.
The only way to do this without code duplication (but with generated
code duplication) is to template the byAction function on the type of
`this`:
auto byAction(this This)() { /* same implementation */ }
Note that this ONLY works if your base range type is an array. If you
have a custom range type, you need to parameterize that based on the
constness of `This`.
-Steve
Re: Feedback to stdout when a specific unittest is run
On 7/23/20 11:10 AM, Per Nordlöw wrote: Is it possible to activate some feedback print to stdout when a unittest is started and ended to track which unittests that take the longest to execute? Alternatively inject a hook being run before each test is run. Yes, you can copy the code [1], and handle the unittest runtime yourself [2], putting in whatever outputs you wish. -Steve [1] https://github.com/dlang/druntime/blob/d19c70b6130d70e6d727cd573230e4519eb0ef55/src/core/runtime.d#L602-L641 [2] https://dlang.org/phobos/core_runtime.html#.Runtime.extendedModuleUnitTester
Re: Alternative to std.range.choose
On 7/22/20 12:33 AM, James Gray wrote:
Is there a better way to achieve behaviour similar to rangeFuncIf
below? f gives a contrived example of when one might want this. g is
how one might try and achieve the same with std.range.choose.
import std.stdio;
import std.range : only, chain, join, choose;
import std.algorithm : map;
auto rangeFunctIf(alias F1, alias F2)(bool c)
if ( __traits(compiles,F1().chain(F2(
{
return only(true).repeat(c?1:0).map!(x=>F1()).join
.chain(only(true).repeat(c?0:1).map!(x=>F2()).join);
}
auto f(ulong n) {
return (n!=0uL).rangeFuncIf!(()=>only(100/n), ()=>only(0));
}
auto g(ulong n) {
return choose(n!=0uL,only(100/n),only(0));
}
void main()
{
writeln(f(2));
writeln(f(0));
writeln(g(2));
//writeln(g(0)); < runtime error
}
I know this is a contrived example, but choose is not good for such a
thing. It should only be used if the range types are different.
g could be:
auto g(ulong n) {
return only(n != 0L ? 100/n : 0)
}
But in any case, changing choose to use lazy is probably a good answer.
-Steve
Re: miscellaneous array questions...
On 7/21/20 8:34 AM, Adam D. Ruppe wrote: The others aren't wrong about stack size limits playing some role, but the primary reason is that it is a weird hack for @safe, believe it or not. ... I don't recall exactly when this was discussed but it came up in the earlier days of @safe, I'm pretty sure it worked before then. I think this was discussed, but was not the reason for the limitation. The limitation exists even in D1, which is before @safe: https://digitalmars.com/d/1.0/arrays.html#static-arrays I have stressed before that any access of a pointer to a large object in @safe code should also check that the base of the object is not within the null page (this is not currently done). This is the only way to ensure safety. -Steve
Re: std/process.d: nothrow functions which throw (in struct ProcessPipes)
On 7/21/20 8:49 AM, Adam D. Ruppe wrote: On Tuesday, 21 July 2020 at 12:44:23 UTC, Drone1h wrote: Would it be possible to explain this, please ? nothrow only applies to Exception and its children. Error is a different branch. Error means you have a programming error and cannot be caught and recovered (though the compiler allows it anyway, it reserves the right to just abort the program instead of actually going through the try/catch system), so it doesn't count as a normal exception, even though it is still thrown. You are supposed to prevent Errors by fixing the bugs in your code, so in a final build they never happen. We need a way to mark posts like "I got this one", so we don't spend the same time saying the same thing lol. -Steve
Re: std/process.d: nothrow functions which throw (in struct ProcessPipes)
On 7/21/20 8:44 AM, Drone1h wrote:
Hello All,
In phobos/std/process.d, in the ProcessPipes struct, we can see a few
functions which are marked with "nothrow", but which (under some
conditions) throw:
@property File stdout() @safe nothrow
{
if ((_redirectFlags & Redirect.stdout) == 0)
throw new Error("Child process' standard output stream
hasn't "
~"been redirected.");
return _stdout;
}
Would it be possible to explain this, please ? Or point me to some
relevant documentation, please ?
Thank you.
nothrow only pertains to Exceptions, not Errors.
Throwing an Error isn't guaranteed to unwind the stack properly (which
is why it can be done within nothrow), and the program should exit if it
ever happens. An Error is a programming mistake (it should never happen
if you wrote your code correctly), whereas an Exception can happen due
to environmental issues or user input.
-Steve
Re: miscellaneous array questions...
On 7/21/20 7:10 AM, IGotD- wrote:
On Monday, 20 July 2020 at 22:05:35 UTC, WhatMeWorry wrote:
2) "The total size of a static array cannot exceed 16Mb" What limits
this? And with modern systems of 16GB and 32GB, isn't 16Mb excessively
small? (an aside: shouldn't that be 16MB in the reference instead of
16Mb? that is, Doesn't b = bits and B = bytes)
I didn't know this but it makes sense and I guess this is a constraint
of the D language itself. In practice 16MB should be well enough for
most cases. I'm not sure where 16MB is taken from, if there is any OS
out there that has this limitation or if it was just taken as an
adequate limit.
I believe it stems from a limitation in the way the stacks are
allocated? Or maybe a limitation in DMC, the basis for DMD.
Also, you CAN actually have larger arrays, they just cannot be put on
the stack (which most static arrays are):
struct S
{
ubyte[17_000_000] big;
}
void main()
{
auto s = new S; // ok
S s; // crash (signal 11 on run.dlang.io)
}
This may not work if `big` had a static initializer, I'm not sure.
-Steve
Re: Accurately serializing and deserializing a SysTime in binary format
On 7/21/20 7:44 AM, Ecstatic Coder wrote:
On Tuesday, 21 July 2020 at 11:01:20 UTC, drug wrote:
On 7/20/20 10:04 PM, Ecstatic Coder wrote:
I'm currently implementing a small open source backup tool (dub), and
therefore I need to accurately store the file modification SysTime in
binary format, so that I can later load this SysTime from the
snapshot file to compare it with the current file modification SysTime.
Having unfortunately not understood how to do this from the SysTime
documentation, in despair, I've tried to directly serialize the 16
bytes of the SysTime value. This worked fine until I call the
".toISOString()" on the deserialized SysTime, which inevitably
crashes the executable ;)
That is probably a bug. I serialize SysTime as long by means msgpack
for exchanging between C++ client and D server and it works pretty nice.
Ah thanks for telling me :)
The loaded byte array in the union type was indeed the same as the saved
one, so I immediately thought it was crashing because of some hidden
pointer for timezone or something which was then pointing to garbage at
reloading, causing the crash of the ".toISOString" call.
Not a bug.
8 of those 16 bytes is a pointer to the timezone, which is going to be
different on different processes.
What you should do I think is serialize the stdTime [1], and set the
time zone to whatever you want:
long serialize(SysTime st) { return st.stdTime; }
SysTime deserialize(long st) { return SysTime(st, UTC()); }
The stdTime is always stored as UTC to make math a lot easier. The time
zone is only used for display.
-Steve
[1] https://dlang.org/phobos/std_datetime_systime.html#.SysTime.stdTime
Re: std.process - avoid interaction with parent shell
On 7/20/20 6:04 PM, Vladimir Panteleev wrote: On Monday, 20 July 2020 at 20:55:52 UTC, Steven Schveighoffer wrote: I tried redirecting /dev/null to stdin when executing my application (and I assumed that would pass onto the process child), but it still asks. What am I doing wrong? Generically, I think you want to detach the program from the current terminal (as well as doing the above). I think setsid can be used for this purpose. Would be a good option possibly to put into std.process. Specifically, checking git's source code, I see that setting the environment variable GIT_TERMINAL_PROMPT to 0 will disable password prompts. Thanks to you and FreeSlave, that seems to work. I had already solved it by just quitting the terminal with it still running. But good to know for the future! -Steve
Re: std.process - avoid interaction with parent shell
On 7/20/20 5:24 PM, H. S. Teoh wrote: On Mon, Jul 20, 2020 at 04:55:52PM -0400, Steven Schveighoffer via Digitalmars-d-learn wrote: I am doing some scripting via D, and using std.process.execute to git clone things. I don't want any user interaction. Occasionally, I get a repository that no longer exists (404). Then git comes up and asks for a username/password. I want it to just fail. Apparently git has no option to be non-interactive, it supposedly checks stdin to see if it's a tty, and only errors if it's not. Try --no-pager perhaps? Not sure if that would help, since this isn't technically a pager that's prompting you. Another way is to take a look at std.process.execute's implementation. I believe it's just a wrapper around spawnProcess. What you want is to adapt that implementation so that it closes stdin before fork-n-exec'ing git; that should stop any prompts. I ran the git command from the shell directly with < /dev/null and it still can ask for username/password. I don't know if it's possible to prevent it. One thing to be aware of is that it may not necessarily be git itself that's prompting you; it could be a helper program like a password manager that creates the prompt. In that case you probably have to find out what it is, and disable it somehow (usually by overriding some environment variable that gets passed to the git child process). I think you might be right. I don't know how it's accessing my terminal, but clearly it can keep doing so even without any handles open. I'm even using ctrl-D and it continues to come up with prompts and wait for input. I was able to solve it by backgrounding the process, and then quitting the parent shell, then it had no option but to error ;) I'm still interested in knowing how this works, if anyone knows. Searching for things like "how does git access my terminal when I closed stdin" doesn't give me much information. -Steve
std.process - avoid interaction with parent shell
I am doing some scripting via D, and using std.process.execute to git clone things. I don't want any user interaction. Occasionally, I get a repository that no longer exists (404). Then git comes up and asks for a username/password. I want it to just fail. Apparently git has no option to be non-interactive, it supposedly checks stdin to see if it's a tty, and only errors if it's not. I tried redirecting /dev/null to stdin when executing my application (and I assumed that would pass onto the process child), but it still asks. What am I doing wrong? e.g.: myapp < /dev/null Username for 'https://github.com': Program pauses, waits for my input. -Steve
Re: alias restriction??!
On 7/19/20 4:21 PM, Carl Sturtivant wrote: On Sunday, 19 July 2020 at 17:06:14 UTC, Paul Backus wrote: Also, letting aliases refer to expressions essentially allows AST macros in through the back door. Consider the following example: [...] Perhaps what's needed is something more that is less than allowing aliases for expressions in the wide sense you suggest here. I agree. Something not yet mentioned is that aliases provide direct access to the symbols for the purposes of looking at attributes -- something that a wrapper function doesn't provide. The question is: how do you restrict it to explicit data items within a specific aggregate without parsing arbitrary expressions? -Steve
Re: vibe.d and my first web service
On Saturday, 18 July 2020 at 09:10:04 UTC, Mr. Backup wrote: by ctrl + c and start again the program cannot start again with error message: Failed to listen on ::1:8080 Failed to listen on 127.0.0.1:8080 Failed to listen for incoming HTTP connections on any of the supplied interfaces. https://github.com/vibe-d/vibe-core/issues/205 -Steve
Re: getopt: How does arraySep work?
On 7/16/20 1:13 PM, Andre Pany wrote: On Thursday, 16 July 2020 at 05:03:36 UTC, Jon Degenhardt wrote: On Wednesday, 15 July 2020 at 07:12:35 UTC, Andre Pany wrote: [...] An enhancement is likely to hit some corner-cases involving list termination requiring choices that are not fully generic. Any time a legal list value looks like a legal option. Perhaps the most important case is single digit numeric options like '-1', '-2'. These are legal short form options, and there are programs that use them. They are also somewhat common numeric values to include in command lines inputs. [...] My naive implementation would be that any dash would stop the list of multiple values. If you want to have a value containing a space or a dash, you enclose it with double quotes in the terminal. Enclose with double quotes in the terminal does nothing: myapp --modelicalibs "file-a.mo" "file-b.mo" will give you EXACTLY the same string[] args as: myapp --modelicalibs file-a.mo file-b.mo I think Jon's point is that it's difficult to distinguish where an array list ends if you get the parameters as separate items. Like: myapp --numbers 1 2 3 -5 -6 Is that numbers=> [1, 2, 3, -5, -6] or is it numbers=> [1, 2, 3], 5 => true, 6 => true This is probably why the code doesn't support that. -Steve
Re: getopt: How does arraySep work?
On 7/14/20 10:22 AM, Steven Schveighoffer wrote: The documentation needs updating, it should say "parameters are added sequentially" or something like that, instead of "separation by whitespace". https://github.com/dlang/phobos/pull/7557 -Steve
Re: getopt: How does arraySep work?
On 7/14/20 10:05 AM, Steven Schveighoffer wrote: Hm... that looks like it IS actually expecting to do what Andre wants. It's adding each successive parameter. If that doesn't work, then there's something wrong with the logic that decides whether a parameter is part of the previous argument or not. Please file a bug. Belay that, the behavior is as designed, I think the issue is the documentation. If arraySep is "", then it's not "separation by whitespace", but rather you must repeat the parameter and each one is appended to the array: dmd -run sample.d --modelicalibs a --modelicalibs b If you want to specify all the parameters in one, you have to provide an arraySep. The documentation needs updating, it should say "parameters are added sequentially" or something like that, instead of "separation by whitespace". -Steve
Re: getopt: How does arraySep work?
On 7/14/20 9:51 AM, Anonymouse wrote:
On Tuesday, 14 July 2020 at 11:12:06 UTC, Andre Pany wrote:
[...]
Steven Schveighoffer already answered while I was composing this, so
discarding top half.
As far as I can tell the default arraySep of "" splitting the argument
by whitespace is simply not the case.
https://github.com/dlang/phobos/blob/master/std/getopt.d#L923
// ...
else static if (isArray!(typeof(*receiver)))
{
// array receiver
import std.range : ElementEncodingType;
alias E = ElementEncodingType!(typeof(*receiver));
if (arraySep == "")
{
*receiver ~= to!E(val);
}
else
{
foreach (elem; val.splitter(arraySep).map!(a => to!E(a))())
*receiver ~= elem;
}
}
So you will probably want an arraySep of " " if you want --modelicalibs
"a b".
Hm... that looks like it IS actually expecting to do what Andre wants.
It's adding each successive parameter.
If that doesn't work, then there's something wrong with the logic that
decides whether a parameter is part of the previous argument or not.
Please file a bug.
-Steve
