Re: Memory overhead for a fiber?
W dniu 06.11.2012 04:43, Nick Sabalausky pisze: What kind of memory overhead is there for a fiber? At least one PAGESIZE (typically 4KB): https://github.com/D-Programming-Language/druntime/blob/master/src/core/thread.d#L3828
Re: Save JSONValue binary in file?
Chopin wrote: Thanks! I tried using it: auto document = parseJSON(content).array; // this works with std.json :) Using json.d from the link: auto j = JSONReader!string(content); auto document = j.value.whole.array; // this doesn't "Error: undefined identifier 'array'" If you're sure that content is an array: auto j = JSONReader!string(content); auto jv = j.value.whole; assert(jv.type == JSONType.array); auto jsonArray = jv.as!(JSONValue[]); alternatively you can replace last line with alias JSONValue[] JSONArray; auto jsonArray = jv.as!JSONArray;
Re: Save JSONValue binary in file?
Chopin wrote: Hello! I got this 109 MB json file that I read... and it takes over 32 seconds for parseJSON() to finish it. So I was wondering if it was a way to save it as binary or something like that so I can read it super fast? Thanks for all suggestions :) Try this implementation: https://github.com/pszturmaj/json-streaming-parser, you can parse all to memory or do streaming style parsing.
Re: Why are scope variables being deprecated?
Jonathan M Davis wrote:
On Thursday, October 11, 2012 01:24:40 Piotr Szturmaj wrote:
Could you give me an example of preventing closure allocation? I think I
knew one but I don't remember now...
Any time that a delegate parameter is marked as scope, the compiler will skip
allocating a closure. Otherwise, it has to copy the stack from the caller onto
the heap to create a closure so that the delegate will continue to work once
the caller has completed (e.g. if the delegate were saved for a callback and
then called way later in the program). Otherwise, it would refer to an invalid
stack and really nasty things would happen when the delegate was called later.
>
By marking the delegate as scope, you're telling the compiler that it will not
escape the function that it's being passed to, so the compiler then knows that
the stack that it refers to will be valid for the duration of that delegate's
existence, so it knows that a closure is not required, so it doesn't allocate
it, gaining you efficiency.
Thanks, that's clear now, but I found a bug:
__gshared void delegate() global;
void dgtest(scope void delegate() dg)
{
global = dg; // compiles
}
void dguse()
{
int i;
dgtest({ writeln(i++); });
}
I guess it's a known one.
Re: Why are scope variables being deprecated?
Jonathan M Davis wrote:
On Wednesday, October 10, 2012 17:04:41 Piotr Szturmaj wrote:
Jonathan M Davis wrote:
On Thursday, July 26, 2012 21:09:09 Chad J wrote:
I keep hearing that scope variables are going away. I missed the
discussion on it. Why is this happening?
When I read about this, I have these in mind:
void someFunc()
{
// foo is very likely to get stack allocated
scope foo = new SomeClass();
foo.use();
// ~foo is called.
}
It's inherently unsafe. What happens if you returned a reference to foo
from someFunc? Or if you assigned a reference to foo to anything and then
tried to use it after someFunc has returned?
Why scope parameters are not deprecated then? It's the same situation.
No. scope on parameters is completely different from scope on local variables.
scope on local variables puts the variable on the stack - even if it's a
class.
Yes, I know the difference between scope parameters and variables, but I
thought that they both can be considered "scope references" which can't
be escaped.
I don't support restoring scope variables in their previous state. But I
think I know a way to make scope variables safe by default.
scope on function parameters is supposed to make it so that the compiler
prevents any references escaping (which potentially really restricts how you
can use the parameter). The only case where that would affect where a variable
is placed is that it makes it so that a closure isn't created for delegates
(which is the one place that scope on parameters actually works semi-
properly). So, the two uses of scope do completely different things.
Could you give me an example of preventing closure allocation? I think I
knew one but I don't remember now...
With regards to escaping scope reference parameters, I hope that
eventually they all will be blocked by the compiler, not only
delegate/closure case.
If you really need foo to be on the stack, then maybe
you should make it a struct.
Then you lose some useful class features.
What you lose is polymorphism, which doesn't work on the stack anyway.
Polymorphism is only applicable when you have a reference which could be of a
base class type rather than the derived type that the object actually is.
Objects on the stack must be their exact type.
I know, class on the stack really become a "value" type. But it's still
useful. You can use non-scope classes with polymorhism as usual, but
when needed you can allocate one concrete class on the stack. You can't
assign subclass reference to scope class variable, but you still can
assign scope class reference to non-scope ancestor class references.
This may or may _not_ escape. I'm proposing that escaping assignments
should be blocked.
scope on local variables is going away for pretty much the same reason
that
delete is. They're unsafe, and the fact that they're in the core language
encourages their use.
That's not convincing for me. Pointers are also unsafe, and they're in
the core language.
Pointers aren't unsafe. Certain operations are unsafe. Note that pointers are
perfectly legal in @safe code. It's pointer arithmetic which isn't.
OK.
and this compiles (http://dpaste.dzfl.pl/6c078e66). With scope storage
class compiler would prevent this escaping assignment. It seems that we
ended up with a solution that was meant to fix a language builtin but
appears to be worse than that.
It may very well be more dangerous, and that may or may not be fixable, but if
it's at the language level, then a lot more people are likely to use it, and
it's dangerous no matter where it is and shouldn't be used under normal
circumstances. Providing the feature is one thing. Making it easy to use is
another. It's like delete. It's dangerous and shouldn't be used normally, so
having it in the language where everyone will use it is too dangerous, so a
library solution is used instead. It therefore becomes more of a power user
feature (as it should be).
I agree about delete operator, but as I wrote above, I'm not sure, but I
might know a way to make scope variables safe. I need to think about this :)
But regardless of the various pros and cons, it was decided ages ago that
it was not worth have scope on local variable be part of the language any
more. So, it's definitely going away.
I see, but scope might be also used in other scenarios, like emplacing
classes inside other classes.
Re: Why are scope variables being deprecated?
bearophile wrote: Piotr Szturmaj: It seems that we ended up with a solution that was meant to fix a language builtin but appears to be worse than that. This is true, currently the library solution is worse (more dangerous and more broken) than the precedent built-in feature. But there is hope to have a good solution someday (mixing library code and some kind of built-support), while a broken built-in is not good. Andrei did the right thing: if you don't have a feature it's kind of easy to add something, while fixing some bad built-in is rather harder. Wasn't it broken because preventing escaping of scoped references was not implemented?
Re: Why are scope variables being deprecated?
Jonathan M Davis wrote:
On Thursday, July 26, 2012 21:09:09 Chad J wrote:
I keep hearing that scope variables are going away. I missed the
discussion on it. Why is this happening?
When I read about this, I have these in mind:
void someFunc()
{
// foo is very likely to get stack allocated
scope foo = new SomeClass();
foo.use();
// ~foo is called.
}
It's inherently unsafe. What happens if you returned a reference to foo from
someFunc? Or if you assigned a reference to foo to anything and then tried to
use it after someFunc has returned?
Why scope parameters are not deprecated then? It's the same situation.
You get undefined behavior, because foo
doesn't exist anymore.
Excuse me, but no, compiler should prevent escaping scope references
just like it does with scope parameters (I know it's currently
implemented just for delegates).
If you really need foo to be on the stack, then maybe
you should make it a struct.
Then you lose some useful class features.
scope on local variables is going away for pretty much the same reason that
delete is. They're unsafe, and the fact that they're in the core language
encourages their use.
That's not convincing for me. Pointers are also unsafe, and they're in
the core language.
> However, if you really do need scope for some
> reason, then you can use std.typecons.scoped, and it'll do the same
thing.
scoped is more dangerous than language solution. See:
class A { }
__gshared A globalA;
static this()
{
auto a = scoped!A;
globalA = a;
}
and this compiles (http://dpaste.dzfl.pl/6c078e66). With scope storage
class compiler would prevent this escaping assignment. It seems that we
ended up with a solution that was meant to fix a language builtin but
appears to be worse than that.
Re: CFTE+DevIL=?
Zhenya wrote: Hi! I need to load some textures for my game,but I woud like to do it in compile time. I know that CTFE imposes restrictions on functions.So can I execute some DevIL(Derelict3) functions? CTFE can only evaluate D functions that have their bodies available. It means that external code coming from C or D external library can't be evaluated.
Re: is array an InputRange?
ref2401 wrote: import std.range; int[] numbers = [1, 3, 5, 7, 9, 11]; auto rangeObject = inputRangeObject(numbers); auto inputRange = cast(InputRange!(int[]))rangeObject; why does 'inputRange' equal null? Because you specified int[] element type and cast operator returns null when objects don't implement requested interface. You need to specify int as element type: auto inputRange = cast(InputRange!int)rangeObject; Anyway you don't need to cast to InputRange(E) interface. You can use rangeObject directly.
Re: inout functions
Timon Gehr wrote:
On 08/24/2012 12:14 AM, Piotr Szturmaj wrote:
Hi,
I found this code of std.range.iota's Result struct:
@property inout(Value) front() inout { assert(!empty); return
current; }
What's the purpose of inout on parameterless functions?
It is a method of a struct, therefore it is not parameterless, but has
a hidden parameter. 'inout' qualifies the implicit 'this' reference.
Inside the method body, typeof(this) is inout(Result).
Thank you. So, it's helpful because struct might be qualified somewhere
as const or as immutable. Anyway in that particular case it's
unnecessary because iota's Result must be mutable to call popFront().
immutable iotaRange = iota(0, 5);
pragma(msg, typeof(&iotaRange.front));
pragma(msg, typeof(iotaRange.front));
io.popFront();
yields:
inout(int) delegate() inout @property
immutable(int)
main.d(61): Error: function
std.range.iota!(int,int).iota.Result.popFront () is not callable using
argument types () immutable
inout functions
Hi,
I found this code of std.range.iota's Result struct:
@property inout(Value) front() inout { assert(!empty); return
current; }
What's the purpose of inout on parameterless functions?
Re: Calling delegate properties without parens
Jonathan M Davis wrote:
On Saturday, April 14, 2012 20:47:20 Piotr Szturmaj wrote:
struct CommonInputRange(E)
{
@property bool delegate() empty;
@property E delegate() front;
void delegate() popFront;
}
front returns an element in the range. In your case, it's returning a
delegate, because you have a range of delegates.
Gah. That explains everything... Thanks!
Re: Calling delegate properties without parens
Artur Skawina wrote:
On 04/15/12 03:01, Piotr Szturmaj wrote:
Artur Skawina wrote:
@property is for functions masquerading as data, i'm not sure extending it
to pointers and delegates would be a good idea. What you are asking for is
basically syntax sugar for:
struct CommonInputRange(E)
{
bool delegate() _empty;
@property auto empty() { return _empty(); };
@property auto empty(typeof(_empty) dg) { _empty = dg; };
E delegate() _front;
@property auto front() { return _front(); };
@property auto front(typeof(_front) dg) { _front = dg; };
void delegate() popFront;
}
Yes, I was thinking about this, but it adds unnecessary overhead. I want to
call delegates directly.
The compiler has to implement it internally exactly like that anyway. D's design
relies on such code being efficient - there is no preprocessor, no inline
attribute and no macros. The trivial functions have to be inlined, if that
doesn't happen it's a compiler bug. Once inlined, there's no overhead.
I wondered if properties can be inlined that way. But you have conviced
me that indeed, inlining should help here. So, I'll use proxy
properties. Thanks for your advice.
Re: Calling delegate properties without parens
Artur Skawina wrote:
@property is for functions masquerading as data, i'm not sure extending it
to pointers and delegates would be a good idea. What you are asking for is
basically syntax sugar for:
struct CommonInputRange(E)
{
bool delegate() _empty;
@property auto empty() { return _empty(); };
@property auto empty(typeof(_empty) dg) { _empty = dg; };
E delegate() _front;
@property auto front() { return _front(); };
@property auto front(typeof(_front) dg) { _front = dg; };
void delegate() popFront;
}
>
Yes, I was thinking about this, but it adds unnecessary overhead. I want
to call delegates directly.
I think the whole idea is harmless because semantically, from the user
perspective, delegates and function pointers works just like normal
functions. So, why not?
Calling delegate properties without parens
I have following code:
import std.array, std.range, std.stdio;
struct CommonInputRange(E)
{
@property bool delegate() empty;
@property E delegate() front;
void delegate() popFront;
}
void main(string[] args)
{
alias CommonInputRange!dchar DCRange;
static assert(isInputRange!DCRange);
DCRange dc;
auto dcr = "abcdefg";
auto t = dcr.takeExactly(3);
dc.empty = &t.empty;
dc.front = &t.front;
dc.popFront = &t.popFront;
for ( ; !dc.empty(); dc.popFront())
writeln(dc.front());
}
As you can see in the for loop, range primitives must be called using
parens (), otherwise they don't work.
Do you know if there are plans to implement @property for delegates and
function pointers?
Re: GUI library
Kevin Cox wrote: I would reccomend Qt as well. You will get native cross-platform widgets with great performance. I am not sure how far QtD is but I know it once had a lot of development on it. AFAIR, QtD is at the alpha stage. It's based on QtJambi, but there is another SMOKE generator, which might be worth giving a try.
Re: std.json dynamic initialization of JSONValue
Nicolas Silva wrote: On Sat, Mar 31, 2012 at 12:25 PM, Piotr Szturmaj wrote: I have written streaming json parser using ranges. It returns slices when possible. Benchmarked it and it's about 2.05x the speed of std.json. It gives possibility to "dig" into the structure and stream (using ranges) by member fields, array elements, or characters of field names and string values. It's possible to parse JSON without a single allocation. For convenience, one can get objects, arrays and strings as a whole. I plan to add a streaming json writer and release it (currently it outputs json using toString()). I've seen questions on stackoverflow about parsing 500 MB JSON... so streaming feature makes it really universal. This approach should be good for XML parser too. Currently, I don't have time to improve it. But if someone is interested I can send it as is :-) I'm very interested in your json lib. I just started writing my own but you have more advanced stuff already so I'd better build something on top of your work. Is it on a repository somewhere on the web? I just uploaded it here: https://github.com/pszturmaj/json-streaming-parser. Hope you like it :-)
Re: std.json dynamic initialization of JSONValue
Andrej Mitrovic wrote: On 12/1/11, Kai Meyer wrote: I'm finding std.json extremely well written, with one glaring exception. I'm finding it to be crap. The last time I used it I just kept getting access violations (or was that std.xml? They're both crap when I used them.). ae.json beats its pants off for its simplicity + you get toJson/jsonParse for serialization and a way to skip serializing fields since a recent commit . It's easy to write your own tree-walking routines as well. But whatever works for people. :) I have written streaming json parser using ranges. It returns slices when possible. Benchmarked it and it's about 2.05x the speed of std.json. It gives possibility to "dig" into the structure and stream (using ranges) by member fields, array elements, or characters of field names and string values. It's possible to parse JSON without a single allocation. For convenience, one can get objects, arrays and strings as a whole. I plan to add a streaming json writer and release it (currently it outputs json using toString()). I've seen questions on stackoverflow about parsing 500 MB JSON... so streaming feature makes it really universal. This approach should be good for XML parser too. Currently, I don't have time to improve it. But if someone is interested I can send it as is :-)
Re: typeof(string.front) should be char
Jonathan M Davis wrote:
On Friday, March 02, 2012 20:41:35 Ali Çehreli wrote:
On 03/02/2012 06:30 PM, Piotr Szturmaj wrote:
> Hello,
>
> For this code:
>
> auto c = "test"c;
> auto w = "test"w;
> auto d = "test"d;
> pragma(msg, typeof(c.front));
> pragma(msg, typeof(w.front));
> pragma(msg, typeof(d.front));
>
> compiler prints:
>
> dchar
> dchar
> immutable(dchar)
>
> IMO it should print this:
>
> immutable(char)
> immutable(wchar)
> immutable(dchar)
>
> Is it a bug?
No, that's by design. When used as InputRange ranges, slices of any
character type are exposed as ranges of dchar.
Indeed.
Strings are always treated as ranges of dchar, because it generally makes no
sense to operate on individual chars or wchars. A char is a UTF-8 code unit. A
wchar is a UTF-16 code unit. And a dchar is a UTF-32 code unit. The _only_ one
of those which is guranteed to be a code point is dchar, since in UTF-32, all
code points are a single code unit. If you were to operate on individual chars
or wchars, you'd be operating on pieces of characters rather than whole
characters, which wreaks havoc with unicode.
Now, technically speaking, a code point isn't necessarily a full character,
since you can also combine code points (e.g. adding a subscript to a letter),
and a full character is what's called a grapheme, and unfortunately, at the
moment, Phobos doesn't have a way to operate on graphemes, but operating on
code points is _far_ more correct than operating on code units. It's also more
efficient.
Unfortunately, in order to code completely efficiently with unicode, you have
understand quite a bit about it, which most programmers don't, but by
operating on ranges of code points, Phobos manages to be correct in the
majority of cases.
I know about Unicode, code units/points and their encoding.
So, yes. It's very much on purpose that all strings are treated as ranges of
dchar.
Foreach gives opportunity to handle any string by char, wchar or dchar,
the default dchar is appropriate here, but why for ranges?
I was afraid it is on purpose, because it has some bad consequences. It
breaks genericity when dealing with ranges. Consider a custom range of char:
struct CharRange
{
@property bool empty();
@property char front();
void popFront();
}
typeof(CharRange.front) and ElementType!CharRange both return _char_
while for string they return _dchar_. This discrepancy pushes the range
writer to handle special string cases. I'm currently trying to write
ByDchar range:
template ByDchar(R)
if (isInputRange!R && isSomeChar!(ElementType!R))
{
alias ElementType!R E;
static if (is(E == dchar))
alias R ByDchar;
else static if (is(E == char))
{
struct ByDchar
{
...
}
}
else static if (is(E == wchar))
{
...
}
}
The problem with that range is when it takes a string type, it aliases
this type with itself, because ElementType!R yields dchar. This is why
I'm talking about "bad consequences", I just want to iterate string by
_char_, not _dchar_.
typeof(string.front) should be char
Hello, For this code: auto c = "test"c; auto w = "test"w; auto d = "test"d; pragma(msg, typeof(c.front)); pragma(msg, typeof(w.front)); pragma(msg, typeof(d.front)); compiler prints: dchar dchar immutable(dchar) IMO it should print this: immutable(char) immutable(wchar) immutable(dchar) Is it a bug?
Re: Absolute beginner
Jorge wrote: My first question si very silly: string str = readln() my input is for example 123 how can i convert this to an integer? import std.conv; // then in code: auto i = to!int(str);
Re: Streams vs ranges
Jonathan M Davis wrote: On Friday, January 13, 2012 12:17:06 Piotr Szturmaj wrote: Is there a plan to replace streams with byte ranges? Or should I just use streams? At some point, std.stream will be replace with a range-based API. There has been some discussion on the design, but it hasn't been fully fleshed out, let alone implemented yet. I need to do some binary parsing and I found using ranges is not very comfortable. For example to read an uint I need to: version (LittleEndian) auto r = retro(takeExactly(range, 4)); else auto r = takeExactly(range, 4); uint u; auto ar = (cast(ubyte*)&u)[0 .. 4]; replaceInPlace(ar, 0, 4, r); while with streams its easier: uint u; stream.read(u); version (LittleEndian) u = swapEndian(u); Just because it's a range doesn't mean that there won't be a function allowing you to do something something more like auto val = read!int(range); That sort of thing will have to be discussed and sorted out when the stream API is overhauled. Unfortunately, it's one of those things that seems to be forever on the TODO list. Thanks for clarifying this. Btw. I find those endian conversions little uncomfortable. I'm talking about: version(LittleEndian) swapEndian() instead of directly calling bigEndianToNative() on uint. I know ubyte[4] param is to avoid mistakes, but I think most of the time programmers know what they do, so IMHO these preventions are unnecessary if we convert integers only. I would leave ubyte[4] params as they are and create overloads for integers using regular int params.
Re: CTFE and cast
Don Clugston wrote:
> On 13/01/12 10:01, k2 wrote:
>> When replace typedef to enum, it became impossible to compile a certain
>> portion.
>>
>> dmd v2.057 Windows
>>
>> enum HANDLE : void* {init = (void*).init}
>>
>> pure HANDLE int_to_HANDLE(int x)
>> {
>> return cast(HANDLE)x;
>> }
>>
>> void bar()
>> {
>> HANDLE a = cast(HANDLE)1;// ok
>> HANDLE b = int_to_HANDLE(2);// ok
>> }
>>
>> HANDLE c = cast(HANDLE)3;// ok
>> HANDLE d = int_to_HANDLE(4);// NG
>>
>> foo.d(17): Error: cannot implicitly convert expression (cast(void*)4u)
>> of type void* to HANDLE
>
> It's a problem. Casting integers to pointers is a very unsafe operation,
> and is disallowed in CTFE. There is a special hack, specifically for
> Windows HANDLES, which allows you to cast integers to pointers at
> compile time, but only after they've left CTFE.
Do you plan to support endianness handling at CTFE?
I mean to write something like this at CT:
union U
{
ubyte[4] ar;
uint num;
}
U u;
u.num = 0x04030201;
if (u.ar[0] == 1)
// little endian
else
// big endian
This is needed to support crypto hashing at CT. std.uuid can generate
uuids based on strings, but they must be hashed first.
Streams vs ranges
Is there a plan to replace streams with byte ranges? Or should I just use streams? I need to do some binary parsing and I found using ranges is not very comfortable. For example to read an uint I need to: version (LittleEndian) auto r = retro(takeExactly(range, 4)); else auto r = takeExactly(range, 4); uint u; auto ar = (cast(ubyte*)&u)[0 .. 4]; replaceInPlace(ar, 0, 4, r); while with streams its easier: uint u; stream.read(u); version (LittleEndian) u = swapEndian(u);
Re: WTF! Parallel foreach more slower that normal foreach in multicore CPU ?
Zardoz wrote:
const num = 10;
>
foreach (t; 0..num) {
foreach(i, ref elem; taskPool.parallel(logs, 100)) {
elem = log(i + 1.0);
}
}
I think you just spawned 10 tasks. Look at foreach (t; 0..num).
Enforcing static closure
I want to make a delegate of blocking I/O statement and pass it to a
function. Then it will be called immediately. This delegate never
escapes its creation scope, so I don't want heap closure allocation.
Will compiler create dynamic closure (e.g. with allocation) or static
closure (with pointer to the stack)?
void recv(ubyte[] buf, out size_t len)
{
// block until data is received
}
int numWaiters;
// will that "scope" enforce static closure?
void wait(scope void delegate() dg)
{
numWaiters++;
dg();
numWaiters--;
}
void test()
{
ubyte[] buf = new ubyte[1500];
size_t len;
wait( { recv(buf, len); } );
}
Non-GC threads
Hi, What are the consequences of using non garbage collected threads in D? I want to write a reliable communication protocol, but I don't want suspend this code by the GC. Is there any method to bind allocated memory to thread itself, so it will be freed after thread terminates, but not in the middle of execution? This is important because in the other case, GC could free memory referenced by non-GC thread. Piotr
Re: Difference between stack-allocated class and struct
Mariusz Gliwiński wrote: I'll clarify myself: All i would need is extending - without polymorphism. Containment, can be solution for fields which doesn't annoys so much (although image in auto-generated documentation, just like it's with subclassing, would be nice). Unfortunately, the worse case is about methods, which have to be manually forwarded to contained struct. So, does someone sees any nice solution for method forwarding as described? Should i make use of some mixin's? You can use 'alias this': http://www.digitalmars.com/d/2.0/class.html#AliasThis
Re: auto arr = new int[10];
%u wrote: is there any different b/w: auto arr = new int[10]; arr is dynamic array of int with ten elements and int[10] arr; ? arr is static array of int with ten elements
Re: Adding days to std.datetime.Date
Steven Schveighoffer wrote: On Thu, 07 Apr 2011 15:07:02 -0400, Piotr Szturmaj wrote: Is it possible to add a particular number of days to a Date? I have number of days since 1 Jan 2000 and I want to convert it to Date: int days = read!int; // number of days since 1 Jan 2000 Date x = Date(2000, 1, 1); x.add!"days"(days); Unfortunately add() does not support adding days. Will it be possible in the future or is there another approach? Yes, use core.time.Duration. Duration was moved to core so it could be used in core functions, like Thread.sleep. so x += dur!"days"(days); See: http://www.digitalmars.com/d/2.0/phobos/core_time.html#dur Well, I did find it few mins ago. But std.datetime's doc still states that it provide types to represent durations of time. -Steve Thanks!
Adding days to std.datetime.Date
Is it possible to add a particular number of days to a Date? I have number of days since 1 Jan 2000 and I want to convert it to Date: int days = read!int; // number of days since 1 Jan 2000 Date x = Date(2000, 1, 1); x.add!"days"(days); Unfortunately add() does not support adding days. Will it be possible in the future or is there another approach? Thanks
Re: Static asserts within unittest block
Now it's clear. Thanks to both of you :)
Static asserts within unittest block
I see this is common practice in Phobos. I though static asserts should be checked at each compilation, not only when compiling with unittest. Or is it supposed to shorten compile time for already tested modules? :)
isVariantN - is expression doesn't match
I wrote simple template to test VariantN type (and thus Algebraic and
Variant).
template isVariantN(T)
{
static if (is(T X == VariantN!(N, Types), uint N, Types...))
enum isVariantN = true;
else
enum isVariantN = false;
}
but testing against Variant yields false:
static assert(isVariantN!(Variant)); // fail
Variant is declared as an alias to VariantN:
alias VariantN!(maxSize!(creal, char[], void delegate())) Variant;
which is equivalent to VariantN!(20u) on 32 bit DMD 2.052:
static assert(is(Variant == VariantN!(20u))); // pass
Strange thing is when I set maxSize other than 20u then isVariantN
yields true:
static assert(isVariantN!(VariantN!(21u))); // pass
static assert(isVariantN!(VariantN!(20))); // pass, 20 as int
static assert(isVariantN!(VariantN!(20u))); // fail, same as Variant
If I add at least one type then it passes even for 20u maxSize.
static assert(isVariantN!(VariantN!(21u, int))); // pass
static assert(isVariantN!(VariantN!(20, int))); // pass, 20 as int
static assert(isVariantN!(VariantN!(20u, int))); // pass
What is going on here?
I also have question regarding integral type aliases such as size_t. For
example I can't use size_t in is expression:
template isVariantN(T)
{
static if (is(T X == VariantN!(N, Types), size_t N, Types...))
enum isVariantN = true;
else
enum isVariantN = false;
}
Error: undefined identifier size_t
TIA
Re: Problem with associative arrays
Jesse Phillips wrote:
Piotr Szturmaj Wrote:
Thank you for your very complete answers :)
I was trying to avoid multiple AA key lookups while appending many
elements to dynamic array. It's clear now, that with D2 semantics it's
better to first build an array and then assign it to AA.
What everyone else said, but you can get a pointer with 'in'
void main() {
uint[][uint] aa;
aa[5] = new uint[0];
auto temp = 5 in aa; // copy uint[] reference
*temp ~= 1;
assert(temp.length == 1 && (*temp)[0] == 1); // pass
assert(aa[5].length == 1 && aa[5][0] == 1); // pass
}
Yes, I already used pointers but in other way:
uint[]* temp = &aa[5]; // copy uint[] reference
and it worked the same as using 'in'. However, I wasn't sure it's
completely safe.
Re: Problem with associative arrays
Thank you for your very complete answers :) I was trying to avoid multiple AA key lookups while appending many elements to dynamic array. It's clear now, that with D2 semantics it's better to first build an array and then assign it to AA.
Problem with associative arrays
Shouldn't dynamic array be reference type? uint[][uint] aa; uint[] temp; aa[5] = new uint[0]; temp = aa[5]; // copy uint[] reference temp ~= 1; assert(temp.length == 1 && temp[0] == 1); // pass assert(aa[5].length == 1 && aa[5][0] == 1); // fail Is this a bug?
Templated nested function can't access 'this'
Why this works:
struct Test
{
int read()
{
return 5;
}
int[] readArray()
{
int[] readDim()
{
return [read(), read()];
}
return readDim();
}
}
but after changing nested function to function template, it doesn't:
struct Test
{
int read()
{
return 5;
}
int[] readArray()
{
int[] readDim(T)()
{
return [read(), read()];
}
return readDim!int();
}
}
Error: need 'this' to access member read
TIA
Re: auto declarations
Ellery Newcomer wrote: auto a = 1, b = null; int a = 1, *b = null; The first is accepted by dmd, and it should result in typeof(a) == int and typeof(b) == void*. It is somewhat contradictory to the error message resulting from the second: multiple declarations must have the same type, not int and int* I am skeptical of dmd's permitting the first. Does anyone else see any utility in it? Personally, I like it. In second line you specify int type, and list of *int* variables. In first line you specify list of initialized variables which types should be inferred automatically. I see no reason why first line should not be permitted. If one would need several variables of one type, why he wouldn't specify exact type instead of using 'auto'?
Re: D2 postgresql interface - Phobos2?
How cool. Very glad you're going native D. I've used Npgsql a lot and also the more standard data.sqlclient interface from c# so I'm happy you're modeling after that API. In your "general" API, will you support the more advanced features like creating functions, refcursors, preplanning queries, etc? I plan to support most of postgres features. Preplanning or preparing queries is already done using prepare() method of PGCommand just like in Npgsql. Cursors also should be available to users. In case of functions I assume native D functions linked to postgres. These must be compiled to shared library and loaded within server but AFAIK shared library support is not complete in D2 (maybe I'm misinformed?). Also there will be support for other advanced features like asynchronous notifications (using NOTIFY channel, payload; syntax). Also, your base db object looks very usefull. Do you have any sense when you would have code ready for testing purposes (don't take this as pressure, just curious)? Or for others to review? Maybe people like Mandeep or myself could help on the coding front? As I guess a DB interface will be used a lot. Non query prepared statements are already working. For general API, I need to finish query result handling and binary formatting of compound types and arrays. Then I will be working on ORM API. I will post source code when result handling is done :) regards, Piotr
Re: D2 postgresql interface - Phobos2?
I was wondering if there is a postgresql db (D native) interface
available for D2?
Hi,
I'm currently writing one for D2 using postgresql's low level protocol
directly (instead of using libpq). It supports binary formatting, so no
parsing or converting to string/escaping is needed and that should give
proper performance.
When done, I will post source on github.
Also, are there any plans to have a common DB interface in Phobos2?
I also have that in my mind. I designed API based on some experience
with .NET and PHP, and I looked at JDBC APIs. Also I've managed to
create base for ORM, example:
struct City
{
Serial!int id; // auto increment
string name;
mixin PrimaryKey!(id);
mixin Unique!(name);
}
struct Pair
{
int a;
int b;
mixin PrimaryKey!(a, b);
}
enum Axis { x, y, z };
struct User
{
Serial!int id; // auto increment
char[30] user;
string password;
Nullable!Axis axis;
Nullable!(int)[3][3] box; // PG's array elements are nullable
Nullable!(int)[] numbers;
Nullable!(int)[][2] twoLists;
Nullable!int cityId;
int a;
int b;
string tag;
mixin PrimaryKey!(id, password);
mixin Unique!(axis, box);
mixin Unique!(user);
mixin Unique!(password, numbers);
mixin ForeignKey!(cityId, City.id, OnDelete.SetNull, OnUpdate.Cascade,
Match.Simple);
mixin ForeignKey!(a, b, Pair.a, Pair.b);
mixin Map!(user, "login", password, "pass");
mixin Ignore!(tag);
}
And there is DBRow struct template:
struct DBRow(T)
{
private T t;
alias t this;
int insert()
{
...
}
int update()
{
...
}
static T getById(...)
{
...
}
...
}
It can be used like this:
DBRow!User r;
r.name = "user";
r.xxx = ...;
r.a = 5;
r.insert();
// DBRow will automatically generate methods for relations
r.getCity().getUsers();
---
Library automatically generates CREATE TABLE/TYPE strings. DBRow will be
in two versions: typed and untyped. Above case is typed version. Untyped
fields will be accessible by index or field name string.
Regular API is similar to .NET API but it's in D's coding style:
PGConnection conn = new PGConnection;
conn.open([
"host" : "localhost",
"database": "test",
"user" : "postgres",
"password" : "postgres"
]);
auto cmd = new PGCommand(conn, "INSERT INTO tbl (id) VALUES ($1)");
cmd.parameters.add(1, PGType.INT8).value = -1;
cmd.prepare();
cmd.bind();
// after bind we have list of field which will be returned
// in case of INSERT there are no fields
foreach (field; cmd.fields)
writeln(field.index, " - ", field.name, ", ", field.oid);
cmd.executeNonQuery();
cmd.parameters[1].value = long.max;
cmd.bind();
cmd.executeNonQuery();
// there's also nice typed query function
auto result = cmd.executeQuery!User();
foreach(row; result)
{
writeln(row.user); // row is DBRow!User
}
// of couse there is version for untyped DBRow
auto resultUntyped = cmd.executeQuery();
foreach(row; result)
{
writeln(row[1]); // same as above
}
---
I'm still working on it, so please be patient :) Of course I will
appreciate any suggestions :)
regards,
Piotr
Re: D2 postgresql interface - Phobos2?
I was wondering if there is a postgresql db (D native) interface
available for D2?
Hi,
I'm currently writing one for D2 using postgresql's low level protocol
directly (instead of using libpq). It supports binary formatting, so no
parsing or converting to string/escaping is needed and that should give
proper performance.
When done, I will post source on github.
Also, are there any plans to have a common DB interface in Phobos2?
I also have that in my mind. I designed API based on some experience
with .NET and PHP, and I looked at JDBC APIs. Also I've managed to
create base for ORM, example:
struct City
{
Serial!int id; // auto increment
string name;
mixin PrimaryKey!(id);
mixin Unique!(name);
}
struct Pair
{
int a;
int b;
mixin PrimaryKey!(a, b);
}
enum Axis { x, y, z };
struct User
{
Serial!int id; // auto increment
char[30] user;
string password;
Nullable!Axis axis;
Nullable!(int)[3][3] box; // PG's array elements are nullable
Nullable!(int)[] numbers;
Nullable!(int)[][2] twoLists;
Nullable!int cityId;
int a;
int b;
string tag;
mixin PrimaryKey!(id, password);
mixin Unique!(axis, box);
mixin Unique!(user);
mixin Unique!(password, numbers);
mixin ForeignKey!(cityId, City.id, OnDelete.SetNull, OnUpdate.Cascade,
Match.Simple);
mixin ForeignKey!(a, b, Pair.a, Pair.b);
mixin Map!(user, "login", password, "pass");
mixin Ignore!(tag);
}
And there is DBRow struct template:
struct DBRow(T)
{
private T t;
alias t this;
int insert()
{
...
}
int update()
{
...
}
static T getById(...)
{
...
}
...
}
It can be used like this:
DBRow!User r;
r.name = "user";
r.xxx = ...;
r.a = 5;
r.insert();
// DBRow will automatically generate methods for relations
r.getCity().getUsers();
---
Library automatically generates CREATE TABLE/TYPE strings. DBRow will be
in two versions: typed and untyped. Above case is typed version. Untyped
fields will be accessible by index or field name string.
Regular API is similar to .NET API but it's in D's coding style:
PGConnection conn = new PGConnection;
conn.open([
"host" : "localhost",
"database": "test",
"user" : "postgres",
"password" : "postgres"
]);
auto cmd = new PGCommand(conn, "INSERT INTO tbl (id) VALUES ($1)");
cmd.parameters.add(1, PGType.INT8).value = -1;
cmd.prepare();
cmd.bind();
// after bind we have list of field which will be returned
// in case of INSERT there are no fields
foreach (field; cmd.fields)
writeln(field.index, " - ", field.name, ", ", field.oid);
cmd.executeNonQuery();
cmd.parameters[1].value = long.max;
cmd.bind();
cmd.executeNonQuery();
// there's also nice typed query function
auto result = cmd.executeQuery!User();
foreach(row; result)
{
writeln(row.user); // row is DBRow!User
}
// of couse there is version for untyped DBRow
auto resultUntyped = cmd.executeQuery();
foreach(row; result)
{
writeln(row[1]); // same as above
}
---
I'm still working on it, so please be patient :) Of course I will
appreciate any suggestions :)
regards,
Piotr
Re: Template matching and is expression
Simen kjaeraas wrote:
Piotr Szturmaj wrote:
static assert(isNullable!(Nullable!int));
Question is, what I'm doing wrong?
The problem here is that Nullable!T is not a real type. Hence,
Nullable!int is actually
Algebraic!(int,void*). Checking for that apparently does not work as
simply as one might
hope.
Algebraic is also an alias for VariantN (which is real type):
template Algebraic(T...)
{
alias VariantN!(maxSize!(T), T) Algebraic;
}
however this check also yields false:
static if (is(T X == VariantN!(U), U...))
but this works:
static if (is(T == Nullable!int))
so, I was a bit confused...
So, instead you should create a wrapper. This works:
struct Nullable(T) {
Algebraic!(T, void*) Nullable;
alias Nullable this;
}
template isNullable(T) {
static if (is(T X == Nullable!(U), U))
enum isNullable = true;
else
enum isNullable = false;
}
static assert(isNullable!(Nullable!int));
Thanks! I've already used wrappers like that, but I though it's possible
to match aliases directly.
regards,
Piotr
Template matching and is expression
Hello,
I'm using D2.051 and following code:
import std.variant;
template Nullable(T)
{
alias Algebraic!(T, void*) Nullable;
}
template isNullable(T)
{
static if (is(T X == Nullable!U, U))
enum isNullable = true;
else
enum isNullable = false;
}
static assert(isNullable!(Nullable!int));
and above static assert fails. Neither this works:
template isNullable(T : Nullable!U, U)
{
enum isNullable = true;
}
template isNullable(T)
{
enum isNullable = false;
}
static assert(isNullable!(Nullable!int));
Question is, what I'm doing wrong?
