Re: New programming paradigm
On Sunday, 3 June 2018 at 16:36:52 UTC, Simen Kjærås wrote:
On Sunday, 3 June 2018 at 14:57:37 UTC, DigitalDesigns wrote:
On Sunday, 3 June 2018 at 09:52:01 UTC, Malte wrote:
You might want to have a look at
https://wiki.dlang.org/Dynamic_typing
This sounds very similar to what you are doing. I never
really looked into it, because I prefer to know which type is
used and give me errors if I try to do stupid things, but I
think it's a cool idea.
No, this is not what I'm talking about, although maybe it
could be related in some way.
Actually, it sort of is. Your mapEnum is essentially the same
as std.variant.visit
(https://dlang.org/phobos/std_variant#.visit), and
std.variant.Algebraic is the type that encapsulates both the
runtime tag and the void[] containing the data of unknown type.
Now, there may be many important differences - Algebraic
encapsulates the data and tag, which may or may not be what you
want, visit only takes one algebraic argument, mapEnum may be
faster, more or less generic, etc. The idea of converting a
run-time value to a compile-time value is the same, though.
--
Simen
I didn't know that variants had those functions! pretty nice.
Yes, it is similar to what I'm doing. Same principles but just a
little different perspective. I use enums, UDA's, and templates
rather than a Algebraic and delegates.
The difference is that the enum stores only the type information
rather than the variable and the type info that Algebraic stores.
If I were to have know about this before I might have used it
instead and everything would have probably been fine.
The only thing is that the enum version lets me store the type
info separately than with the data. When several variables depend
on the type id I think it will make it a little easier than
having to manage several Algebraic type info's across several
variables to sync them.
For example
dataType type;
void[] in, out;
rather than
Algebraic!(type1,..., typen) in, out;
and then having to make sure the types are synced between in and
out. At least in my case it might be a little easier. Also my way
uses a templated function directly rather than an array of
lambads, although they are equivalent:
Algebraic!(string, int) variant;
variant.visit!((string s) => cast(int) s.length, (int i)=>
i)();
which could be written as
variant.visit!((string s) => foo(s), (int i)=> foo(i))();
auto foo(T)(T t) { }
would become
enum variant
{
@("int") _int,
@("string") _string,
}
mixin(variant.MapEnum!("foo")());
auto foo(T)(T t) { }
So, they definitely are very similar and actually might be
identical. I haven't used Algebraic and visit any to know.
What I do know is that for several Algebraics you would have to
do something like
variant.visit!((string s) => variant2.visit!((double d) => {
foo(s,d); })), (int i)=> foo(i))();
etc. Which is creating the nested switch structure and can become
complicated while my method still remains one line but foo just
takes more than one template parameter. My feeling is mine is a
little less robust since it's more for specific types of code
while visit is a little more general. Mainly because of the hard
coding of the mixin structure.
Re: Confusion/trying to understand CTFE keywords
On Sunday, June 03, 2018 21:32:06 gdelazzari via Digitalmars-d-learn wrote:
> Hello everyone, I'm new here on the forum but I've been exploring
> D for quite a while. I'm not an expert programmer by any means,
> so this one may be a really silly question and, in that case,
> please forgive me.
>
> With the premise that I've still not looked a lot into "complex"
> compile time "stuff" (whether it's templates, CTFE, compile-time
> constants, etc...), so that may be the reason I may be missing
> the point... I'm trying to understand why keywords such as
> "static" or "enum" are used to denote compile time "things". What
> I mean is that those keywords are also used for other purposes,
> so I find it a bit confusing. Couldn't a keyword like "ctfe"
> (just making it up right now) exist? So that, when seeing
> something like
>
> ctfe myNumber = 5;
>
> ctfe if (myNumber + 2 == 7)
> {
>// ...
> }
>
> one could immediately understand that the code is
> executed/evaluated at compile time. True, after someone knows
> that "static" and "enum" mean (in the above example) that some
> compile-time things are happening, it's fine. I just find it a
> bit confusing not having a dedicated keyword but re-using
> existing ones that also serve other purposes...
>
> Note that this is not an attack to the language or anything (I
> actually really love it), I'm just trying to understand the
> reasoning behind this choice.
>
> Thank you very much in advance.
I think that part of your problem here comes from the fact that you think of
enum or static are "CTFE keywords." That's not what they are at all. Yes,
they can trigger CTFE, but they're not the only way.
Given how CTFE works in D, it really wouldn't make sense to have a keyword
for it. CTFE is simply what kicks in when you have an expression that _must_
be evaluated at compile time. e.g.
enum a = 42;
enum Foo
{
a = 42,
b = 92
c = 12
}
struct S
{
int i = 9;
}
all are cases where CTFE is used, because all of them require that the value
be known at compile time. In the example, they're just integers, so no
functions are called, but any one of them could be initialized with an
expression that involved calling a function. e.g.
struct S
{
int i = foo() + 7;
}
There is no special keyword here, and there is no need for one. By the
language's design, if i is directly initialized, its value must be known at
compile time, and so any expression that's used to directly initialize it
must be evaluated at compile time. The same goes for enums or static
variables. How would you expect something like this to even work with a
special keyword for CTFE?
Now, the fact that enum was used for manifest constants such as
enum foo = "hello";
in addition to actual enums such as
enum Color
{
red,
green,
blue,
orange,
yellow
}
is arguably unnecessary and confusing (and as some of the other posts in
this thread mention, this was done to avoid adding a new keyword). So,
maybe, we should have had something like
manifest foo = "hello";
and made
enum foo = "hello";
illegal, but even if we had done something like that, it would not have had
any effect on how CTFE works. It would have just clarified the difference
between manifest constants and proper enums. As for why enum was reused for
manifest constants, it was not only to save a keyword but because they
bascially act like anonymous enums (and in fact, that's what the spec calls
them) in that how they act is exactly like enums except for the fact that
they don't declare a new type. So, whether it would have been better to use
a new keyword is a matter of debate.
As for static, most of what it does is inherited from C/C++ and Java, and
while does get used in several contexts, it's actually used quite
consistantly, much as it might not seem that way at first.
When talking about a static variable, the key difference between a static
variable and an enum (be it a manifest constant or an actual enum) is that a
static variable is an actual variable with a location in memory, whereas an
enum is just a value that you can refer to by name. The value of the enum is
basically copy-pasted wherever it it is used. It's not legal to take the
address of an enum, and in fact, that means that if you do something like
enum arr = [1, 2, 3];
then every time you use arr, you're potentially allocating a new dynamic
array, because foo(arr) is the same as foo([1, 2, 3]) except for two things:
1. If you change the value of arr, it changes its value everywhere, wheres
if you use [1, 2, 3] directly, you'd have to change every place that you
used it if you wanted to change it.
2. The value of arr is copied, not the expression used to initialize it. So,
if you had
enum arr = [1, 2, bar()];
and bar() resulted in 42, then foo(arr) would be the same as foo([1, 2, 42])
and not foo([1, 2, bar()]).
However, static variables are actually variables. The key difference between
them and other variables at the same scope is that
Re: Confusion/trying to understand CTFE keywords
On Sunday, 3 June 2018 at 21:32:06 UTC, gdelazzari wrote: Note that this is not an attack to the language or anything (I actually really love it), I'm just trying to understand the reasoning behind this choice. Because they have a thing about not adding new keywords, apparently it's more important that people dont lose a word they can use for a variable name than it is for expressive non context dependent keywords.
Re: Confusion/trying to understand CTFE keywords
On Sunday, 3 June 2018 at 21:32:06 UTC, gdelazzari wrote: I'm trying to understand why keywords such as "static" or "enum" are used to denote compile time "things". What I mean is that those keywords are also used for other purposes, so I find it a bit confusing. Couldn't a keyword like "ctfe" (just making it up right now) exist? I believe the enum was chosen over a dedicated keyword for compile-time constants to prevent adding another keyword, as D already has quite a few.
Confusion/trying to understand CTFE keywords
Hello everyone, I'm new here on the forum but I've been exploring
D for quite a while. I'm not an expert programmer by any means,
so this one may be a really silly question and, in that case,
please forgive me.
With the premise that I've still not looked a lot into "complex"
compile time "stuff" (whether it's templates, CTFE, compile-time
constants, etc...), so that may be the reason I may be missing
the point... I'm trying to understand why keywords such as
"static" or "enum" are used to denote compile time "things". What
I mean is that those keywords are also used for other purposes,
so I find it a bit confusing. Couldn't a keyword like "ctfe"
(just making it up right now) exist? So that, when seeing
something like
ctfe myNumber = 5;
ctfe if (myNumber + 2 == 7)
{
// ...
}
one could immediately understand that the code is
executed/evaluated at compile time. True, after someone knows
that "static" and "enum" mean (in the above example) that some
compile-time things are happening, it's fine. I just find it a
bit confusing not having a dedicated keyword but re-using
existing ones that also serve other purposes...
Note that this is not an attack to the language or anything (I
actually really love it), I'm just trying to understand the
reasoning behind this choice.
Thank you very much in advance.
Re: how to define infix function
On Saturday, 2 June 2018 at 23:17:48 UTC, Simen Kjærås wrote:
On Saturday, 2 June 2018 at 22:09:49 UTC, Neia Neutuladh wrote:
On Saturday, 2 June 2018 at 21:44:39 UTC, greatsam4sure wrote:
Sorry for the typo
is it possible to define infix function in D
3.min(5)// 3: where min is a function, works in D
3 min 5 // does not work.
thanks in advance
This is a horrible abuse of D's operator overloading
discovered by FeepingCreature in the distant past.
You have to delimit your custom infix operator with slashes;
you can't make `3 min 5` work, but you can make `3 /min/ 5`
work.
Observe:
struct Min
{
MinIntermediate!T opBinaryRight(string op, T)(T value) if
(op == "/")
{
return MinIntermediate!T(value);
}
}
struct MinIntermediate(T)
{
T value;
T opBinary(string op, T)(T value2) if (op == "/")
{
if (value < value2) return value;
return value2;
}
}
Min min;
void main()
{
writeln(1 /min/ 2);
}
And of course, this can be generalized:
struct Operator(alias fn, string operator = "/")
{
static auto opBinaryRight(string op : operator, T...)(T
value1)
{
struct Result
{
auto opBinary(string op : operator, U...)(U value2)
if (__traits(compiles, fn(value1, value2)))
{
return fn(value1, value2);
}
}
Result result;
return result;
}
}
unittest
{
import std.algorithm.comparison;
alias min = Operator!(std.algorithm.comparison.min);
assert(1 /min/ 3 == 1);
}
Note also the use of static opBinaryRight, allowing one to
eschew the 'min' variable.
All of this said, I would suggest not using this in prod - it's
a neat trick that shows off some of D's power, but I don't see
a case where this would be easier to understand than a
straightforward function call.
--
Simen
That is interesting. I don't know yet whether it's good or bad,
but certainly, it's interesting.
Re: New programming paradigm
On Sunday, 3 June 2018 at 14:57:37 UTC, DigitalDesigns wrote: On Sunday, 3 June 2018 at 09:52:01 UTC, Malte wrote: You might want to have a look at https://wiki.dlang.org/Dynamic_typing This sounds very similar to what you are doing. I never really looked into it, because I prefer to know which type is used and give me errors if I try to do stupid things, but I think it's a cool idea. No, this is not what I'm talking about, although maybe it could be related in some way. Actually, it sort of is. Your mapEnum is essentially the same as std.variant.visit (https://dlang.org/phobos/std_variant#.visit), and std.variant.Algebraic is the type that encapsulates both the runtime tag and the void[] containing the data of unknown type. Now, there may be many important differences - Algebraic encapsulates the data and tag, which may or may not be what you want, visit only takes one algebraic argument, mapEnum may be faster, more or less generic, etc. The idea of converting a run-time value to a compile-time value is the same, though. -- Simen
Re: Line endings when redirecting output to file on windows.
On Sunday, 3 June 2018 at 15:42:48 UTC, rikki cattermole wrote: On 04/06/2018 3:24 AM, Bastiaan Veelo wrote: I need some help understanding where extra '\r' come from when output is redirected to file on Windows. First, this works correctly: rdmd --eval="(\"hello\" ~ newline).toFile(\"out.txt\");" As expected, out.txt contains "hello\r\n". I would expect the following to do the same, but it doesn't: rdmd --eval="write(\"hello\" ~ newline);" > out.txt Now out.txt contains "hello\r\r\n". Who is doing the extra conversion here, and how do I stop it? Thanks! Bastiaan. That would be cmd. Not sure you can stop it without piping it after rdmd to remove the \r. Thanks. It is starting to dawn on me that I shouldn't use `newline` and `toFile` to write text files, but rather always use "\n" as line ending and use `write` for both writing to stdout and file. rdmd --eval="File(\"out.txt\", \"w\").write(\"hello\n\");" and rdmd --eval="write(\"hello\n\");" > out.txt both produce "hello\r\n" on Windows. Am I correct, or is there a more idiomatic way of writing strings to text files?
Re: pipeProcess failing
On Sunday, 3 June 2018 at 15:07:07 UTC, DigitalDesigns wrote:
I'm calling pipe process using
pipeProcess([AliasSeq!args], Redirect.stdout | Redirect.stdin);
where args is a tuple.
Everything works when I pass each argument individually. If I
combine any args using a space it fails or if I pass an
argument with "".
So I guess something like this
pipeProcess(["dmd", "", "-m32 -JC:\"], Redirect.stdout |
Redirect.stdin);
will fail while
pipeProcess(["dmd", "-m32", "-JC:\"], Redirect.stdout |
Redirect.stdin);
works.
Is this a bug or something else going on I'm not aware of?
I'm just wrapping pipe process in a function foo(Args...)(Args
args) and calling it like foo("dmd", "", "-m32 -JC:\").
The reason why it is a problem is that it will simplify some
code to be able to combine some arguments.
The argument list is just passed along to the child process as
its argv. So if the child process is able to handle empty strings
or space-combined arguments in argv, then it will work, and if
the child process can't do that, it will fail.
Normally, the command-line shell takes care of splitting
arguments into separate strings, and never passes empty
arguments, so most programs are not prepared to handle those
cases. If you want the shell to do this work for you, you can try
using pipeShell instead of pipeProcess. Otherwise, you will have
to do it yourself.
Re: New programming paradigm
On Monday, 4 September 2017 at 03:26:23 UTC, EntangledQuanta
wrote:
Take a variant type. It contains the "type" and the data. To
simplify, we will treat look at it like
(pseudo-code, use your brain)
enum Type { int, float }
foo(void* Data, Type type);
The normal way to deal with this is a switch:
switch(type)
{
case int: auto val = *(cast(int*)Data);
case float: auto val = *(cast(float*)Data);
}
But what if the switch could be generated for us?
[...]
But, in fact, since we can specialize on the type we don't have
to use switch and in some cases do not even need to specialize:
for example:
foo(T)(T* Data) { writeln(*Data); }
is a compile time template that is called with the correct type
value at run-time due to the "magic" which I have yet to
introduce.
Note that if we just use a standard runtime variant, writeln
would see a variant, not the correct type that Data really is.
This is the key difference and what makes this "technique"
valuable. We can treat our dynamic variables as compile time
types(use the compile time system) without much hassle. They
fit naturally in it and we do not clutter our code switches. We
can have a true auto/var like C# without the overhead of the
IR. The cost, of course, is that switches are still used, they
are generated behind the scenes though and the runtime cost is
a few instructions that all switches have and that we cannot
avoid.
To get a feel for what this new way of dealing with dynamic
types might look like:
void foo(var y) { writeln(y); }
var x = "3"; // or possibly var!(string, int) for the explicit
types used
foo(x);
x = 3;
foo(x);
It sounds like what you are describing is a sum type. There is an
implementation of one in the standard library,
std.variant.Algebraic, as well as several alternative
implementations on code.dlang.org, including my own, "sumtype"
[1].
Using sumtype, your example would look like this:
alias Var = SumType!(string, int);
void foo(Var y) {
var.match!(
(value) { writeln(value); } // template lambda
);
}
Var x = "3";
foo(x);
x = 3;
foo(x);
The match method takes a list of functions as template arguments,
and generates a switch statement that maps each possible type of
Var to one of those functions. All type checking is done at
compile time.
[1] https://code.dlang.org/packages/sumtype
Re: Line endings when redirecting output to file on windows.
On 04/06/2018 3:24 AM, Bastiaan Veelo wrote: I need some help understanding where extra '\r' come from when output is redirected to file on Windows. First, this works correctly: rdmd --eval="(\"hello\" ~ newline).toFile(\"out.txt\");" As expected, out.txt contains "hello\r\n". I would expect the following to do the same, but it doesn't: rdmd --eval="write(\"hello\" ~ newline);" > out.txt Now out.txt contains "hello\r\r\n". Who is doing the extra conversion here, and how do I stop it? Thanks! Bastiaan. That would be cmd. Not sure you can stop it without piping it after rdmd to remove the \r.
Line endings when redirecting output to file on windows.
I need some help understanding where extra '\r' come from when output is redirected to file on Windows. First, this works correctly: rdmd --eval="(\"hello\" ~ newline).toFile(\"out.txt\");" As expected, out.txt contains "hello\r\n". I would expect the following to do the same, but it doesn't: rdmd --eval="write(\"hello\" ~ newline);" > out.txt Now out.txt contains "hello\r\r\n". Who is doing the extra conversion here, and how do I stop it? Thanks! Bastiaan.
pipeProcess failing
I'm calling pipe process using
pipeProcess([AliasSeq!args], Redirect.stdout | Redirect.stdin);
where args is a tuple.
Everything works when I pass each argument individually. If I
combine any args using a space it fails or if I pass an argument
with "".
So I guess something like this
pipeProcess(["dmd", "", "-m32 -JC:\"], Redirect.stdout |
Redirect.stdin);
will fail while
pipeProcess(["dmd", "-m32", "-JC:\"], Redirect.stdout |
Redirect.stdin);
works.
Is this a bug or something else going on I'm not aware of?
I'm just wrapping pipe process in a function foo(Args...)(Args
args) and calling it like foo("dmd", "", "-m32 -JC:\").
The reason why it is a problem is that it will simplify some code
to be able to combine some arguments.
Re: New programming paradigm
On Sunday, 3 June 2018 at 09:52:01 UTC, Malte wrote:
On Saturday, 2 June 2018 at 23:12:46 UTC, DigitalDesigns wrote:
On Thursday, 7 September 2017 at 22:53:31 UTC, Biotronic wrote:
[...]
I use something similar where I use structs behaving like
enums. Each field in the struct is an "enum value" which an
attribute, this is because I have not had luck with using
attributes on enum values directly and that structs allow
enums with a bit more power.
[...]
You might want to have a look at
https://wiki.dlang.org/Dynamic_typing
This sounds very similar to what you are doing. I never really
looked into it, because I prefer to know which type is used and
give me errors if I try to do stupid things, but I think it's a
cool idea.
No, this is not what I'm talking about, although maybe it could
be related in some way.
What I am talking about is hooking up a runtime variable that can
take a few values, such as from an enum and have those be mapped
to a compile time template value.
This way you get full static time checking of runtime code. Seems
impossible? It's not!
What it does is leverage D's meta programming engine to deal with
all the routine possiblities.
A large switch statement is what makes the runtime to compile
time magic happen
int x;
switch(x)
{
default: foo!void();
case 0: foo!int();
case 1: foo!double();
etc...
}
See how the switch maps a runtime value x to a templated function
foo?
we then can handle the x values with foo
void foo(T)()
{
// if x = 0 then T = int
// if x = 1 then T = double
}
But inside foo we have T, the template variable that is the
compile time representation of the dynamic variable x. Remember,
x's value is unknown at compile time... the switch is what maps
the runtime to the compile time.
But in foo, because we have T, the type system all works fine.
What makes this very useful that we can call templated functions
using T and the meta engine will pick the right template function
specialization.
e.g.,
void bar(S)()
{
}
can be used inside foo by calling bar!T(). It doesn't seem like
much here if you had to use x it would be a pain. Either you
would have to manually create switches or create a rats nest of
if statements. But we never have to worry about that stuff when
using the above method because it is exactly like programming at
compile time as if x a compile time value(like say, just int).
It works great when you have several template variables and just
want everything to work together without having to go in to much
trouble:
foo(A,B)()
{
B b = 4;
bar!(A)(b)
}
suppose A can come from int, double, and B from float and long
That is 4 different combinations one would normally have to
represent. Not a big deal until you have to handle every
combination.
Suppose you are working with void arrays. They contain types but
you don't know the type except after compile time.
Without using this technique you have to use casts and tricks and
you'll find out if you screwed up some typing stuff at runtime.
Using this technique you will not have a void array but a T[]
with T being any of the possible types that you specify using
UDA's.
if you could
if (x == 0)
{
foo(cast(int[])a);
} else
if (x == 1)
{
foo(cast(double[])a);
} else
but I can do that with one line which simply generates the switch
for me. Really all I'm doing is hiding the switch so it all looks
like some magic is happening in one line. But the fact that it
becomes really simple to do seems to open up the use of it and
conceptually on can then think of "x" as a compile time variable
that can take on several possibilities.
Re: how to define infix function
On Saturday, 2 June 2018 at 22:09:49 UTC, Neia Neutuladh wrote:
On Saturday, 2 June 2018 at 21:44:39 UTC, greatsam4sure wrote:
[...]
This is a horrible abuse of D's operator overloading discovered
by FeepingCreature in the distant past.
You have to delimit your custom infix operator with slashes;
you can't make `3 min 5` work, but you can make `3 /min/ 5`
work.
Observe:
struct Min
{
MinIntermediate!T opBinaryRight(string op, T)(T value) if
(op == "/")
{
return MinIntermediate!T(value);
}
}
struct MinIntermediate(T)
{
T value;
T opBinary(string op, T)(T value2) if (op == "/")
{
if (value < value2) return value;
return value2;
}
}
Min min;
void main()
{
writeln(1 /min/ 2);
}
thanks
Re: New programming paradigm
On Saturday, 2 June 2018 at 23:12:46 UTC, DigitalDesigns wrote: On Thursday, 7 September 2017 at 22:53:31 UTC, Biotronic wrote: [...] I use something similar where I use structs behaving like enums. Each field in the struct is an "enum value" which an attribute, this is because I have not had luck with using attributes on enum values directly and that structs allow enums with a bit more power. [...] You might want to have a look at https://wiki.dlang.org/Dynamic_typing This sounds very similar to what you are doing. I never really looked into it, because I prefer to know which type is used and give me errors if I try to do stupid things, but I think it's a cool idea.
Re: Convert a huge SQL file to CSV
On Friday, 1 June 2018 at 10:15:11 UTC, Martin Tschierschke wrote: On Friday, 1 June 2018 at 09:49:23 UTC, biocyberman wrote: I need to convert a compressed 17GB SQL dump to CSV. A workable solution is to create a temporary mysql database, import the dump, query by python, and export. But i wonder if there is something someway in D to parse the SQL file directly and query and export the data. I imagine this will envolve both parsing and querying because the data is stored in several tables. I am in the process of downloading the dump now so I can’t give excerpt of the data. You don't need python: https://michaelrigart.be/export-directly-mysql-csv/ SELECT field1, field2 FROM table1 INTO OUTFILE '/path/to/file.csv' FIELDS TERMINATED BY ',' ENCLOSED BY '"' FIELDS ESCAPED BY '\' LINES TERMINATED BY '\n'; Most important: INTO OUTFILE : here you state the path where you want MySQL to store the CSV file. Keep in mind that the path needs to be writeable for the MySQL user You can write a parser for SQL in D, but even if the import into mysql would take some time, it's only compute time and not yours. Regards mt. Ah yes, thank you Martin. I forgot that we can do a "batch" SQL query where mysql server can parse and run query commands. So no need for Python. But I am still currently waiting for the import to finish the importing of mysql dump. It took 18 hours and is still counting! The whole mysql database is 68GB at the moment. Can we avoid the import and query the database dump directly?
