I want to transmit the class name and the member name in the method
I think code style like:
~~
struct User
{
int id;
string name;
string email;
}
class ORM
{
}
auto db = new ORM;
auto users =
db.select(User).where(email.like("*@hotmail.com")).limit(10);
foreach(user; users)
{
writeln("user: " + user.name + "\n");
}
~~
this rust code is support it:
https://github.com/diesel-rs/diesel/blob/master/examples/postgres/all_about_updates/src/lib.rs
Re: Help optimizing UnCompress for gzipped files
On 04.01.18 20:46, Steven Schveighoffer wrote: > On 1/4/18 1:57 PM, Christian Köstlin wrote: >> Thanks Steve, >> this runs now faster, I will update the table. > > Still a bit irked that I can't match the C speed :/ > > But, I can't get your C speed to duplicate on my mac even with gcc, so > I'm not sure where to start. I find it interesting that you are not > using any optimization flags for gcc. I guess, the code in my program is small enough that the optimize flags do not matter... most of the stuff is pulled from libz? Which is dynamically linked against /usr/lib/libz.1.dylib. I also cannot understand what I should do more (will try realloc with Mallocator) for the dlang-low-level variant to get to the c speed. rust is doing quite well there -- Christian Köstlin
Re: Consequences of casting away immutable from pointers
On Friday, January 05, 2018 04:16:48 Adam D. Ruppe via Digitalmars-d-learn wrote: > On Friday, 5 January 2018 at 04:10:54 UTC, Steven Schveighoffer > > wrote: > > The compiler assumes x is going to be 5 forever, so instead of > > loading the value at that address, it just loads 5 into a > > register (or maybe it just folds x == 5 into true). > > I was curious what dmd did, and the disassembly indeed shows it > just loads 5 into the register and leaves it there - assuming > since it is immutable, it will never change through any pointer > and thus never reloads it from memory at any time. > > Interestingly, dmd -O just stubs out the whole function. I guess > it assumes all the defined behavior actually accomplishes nothing > and it is free to optimize out undefined behavior... thus the > function needs no code. Similarly, if the last assert is changed > to x != 5, dmd -O doesn't even actually do a comparison (the > value 5 never appears in the generated code!), it just outputs > the direct call to assertion failure. Well, it's certainly nice to see some evidence that the compiler really is taking advantage of the guarantees that immutable is supposed to provide. - Jonathan M Davis
Re: Consequences of casting away immutable from pointers
On Thursday, January 04, 2018 23:10:54 Steven Schveighoffer via Digitalmars- d-learn wrote: > On 1/4/18 10:58 PM, jmh530 wrote: > > I'm trying to understand the consequences of casting away immutable from > > a pointer. The code below has some weird things going on like the > > pointers still point to the correct address but when you dereference > > them they don't point to the correct value anymore. > > > > Should I just assume this is undefined behavior and not bother with it? > > Or is there a use case for this? > > Yes, this is undefined behavior. > > https://dlang.org/spec/const3.html#removing_with_cast > > The compiler assumes x is going to be 5 forever, so instead of loading > the value at that address, it just loads 5 into a register (or maybe it > just folds x == 5 into true). > > The compiler would likely be free to assume *p_x == 5 forever also, if > it was clever enough. > > I'd recommend not doing this. Yeah, casting away either const or immutable is just begging for trouble, though it's likely to be worse with immutable, since there are more optimizations that the compiler can do based on immutable. D's const and immutable are definitely not the same as C++'s const and treating either of them like they have backdoors is just going to cause bugs. If you ever need a backdoor to get around them, then you shouldn't be using them. - Jonathan M Davis
Re: Consequences of casting away immutable from pointers
On Friday, 5 January 2018 at 04:10:54 UTC, Steven Schveighoffer wrote: The compiler assumes x is going to be 5 forever, so instead of loading the value at that address, it just loads 5 into a register (or maybe it just folds x == 5 into true). I was curious what dmd did, and the disassembly indeed shows it just loads 5 into the register and leaves it there - assuming since it is immutable, it will never change through any pointer and thus never reloads it from memory at any time. Interestingly, dmd -O just stubs out the whole function. I guess it assumes all the defined behavior actually accomplishes nothing and it is free to optimize out undefined behavior... thus the function needs no code. Similarly, if the last assert is changed to x != 5, dmd -O doesn't even actually do a comparison (the value 5 never appears in the generated code!), it just outputs the direct call to assertion failure.
Re: Consequences of casting away immutable from pointers
On 1/4/18 10:58 PM, jmh530 wrote: I'm trying to understand the consequences of casting away immutable from a pointer. The code below has some weird things going on like the pointers still point to the correct address but when you dereference them they don't point to the correct value anymore. Should I just assume this is undefined behavior and not bother with it? Or is there a use case for this? Yes, this is undefined behavior. https://dlang.org/spec/const3.html#removing_with_cast The compiler assumes x is going to be 5 forever, so instead of loading the value at that address, it just loads 5 into a register (or maybe it just folds x == 5 into true). The compiler would likely be free to assume *p_x == 5 forever also, if it was clever enough. I'd recommend not doing this. -Steve
Consequences of casting away immutable from pointers
I'm trying to understand the consequences of casting away
immutable from a pointer. The code below has some weird things
going on like the pointers still point to the correct address but
when you dereference them they don't point to the correct value
anymore.
Should I just assume this is undefined behavior and not bother
with it? Or is there a use case for this?
void main()
{
immutable(int) x = 5;
auto p_x =
int* p_x_alt = cast(int*)p_x;
(*p_x_alt)++;
//addresses remain unchanged
assert( == p_x);
assert(p_x == p_x_alt);
assert(*p_x_alt == 6);
assert(*p_x == *p_x_alt); //but p_x and p_x_alt point to same
value
assert(x != *p_x); //yet that is not the case for x
assert(x == 5); //which still is 5
}
Re: Any free stock market data API?
On Friday, 5 January 2018 at 01:45:46 UTC, Laeeth Isharc wrote: On Thursday, 4 January 2018 at 23:04:44 UTC, Amorphorious wrote: Most are in other languages: https://www.alphavantage.co/ https://iextrading.com/ are two free ones. I'm just hoping for a more D'ish solution. I wrote a simple api for quandl.com and somewhere I have one for yahoo. Neither being used right now so might have broken. https://github.com/Laeeth/d-quandl/blob/master/quandl.d I heard Yahoo changed up their stock api earlier this year, I think. I haven't used it in a few years.
Re: Any free stock market data API?
On Thursday, 4 January 2018 at 23:04:44 UTC, Amorphorious wrote: Most are in other languages: https://www.alphavantage.co/ https://iextrading.com/ are two free ones. I'm just hoping for a more D'ish solution. I wrote a simple api for quandl.com and somewhere I have one for yahoo. Neither being used right now so might have broken. https://github.com/Laeeth/d-quandl/blob/master/quandl.d
Re: Any free stock market data API?
On Thursday, 4 January 2018 at 23:04:44 UTC, Amorphorious wrote: Most are in other languages: https://www.alphavantage.co/ https://iextrading.com/ are two free ones. I'm just hoping for a more D'ish solution. http://www.quantmod.com/ I'm wondering if I can use something like alphavantage and this package and integrate it between D and R. Have R/qauntmod do all the statistical work to avoid rewriting it all but easily coordinate all this from D. Or maybe there is a better way?
Any free stock market data API?
Most are in other languages: https://www.alphavantage.co/ https://iextrading.com/ are two free ones. I'm just hoping for a more D'ish solution.
Re: How do you do "const nazi" in D?
On Thursday, 4 January 2018 at 13:52:14 UTC, Mark wrote: You ma find Ali Cehreli's talk from DConf 2013 [1] useful. It includes some guidelines (basically recommendations) on how to use immutable and const. Same topic and same mentor here, but in text form: http://ddili.org/ders/d.en/const_and_immutable.html In my opinion it is an excellent answer to this question.
Re: How do i activate the latest nightly without knowing the date ?
On Thursday, 4 January 2018 at 20:37:16 UTC, user789 wrote: On Thursday, 4 January 2018 at 20:28:47 UTC, user789 wrote: After using the script to setup DMD nightly i read in the output that i have to run "source ~/dlang/dmd-master-2018-01-04/activate" to have DMD DUB etc working. Is there another activation command, working without the date, suitable for a cron job ? "source ~/dlang/dmd-nightly/activate" doesn't work but would be better when you just want to use the nightly F1nally i found that just --- wget https://nightlies.dlang.org/dmd-nightly/dmd.master.linux.tar.xz tar xf ./dmd.master.linux.tar.xz03:12 export PATH=$PATH:`pwd`/dmd2/linux/bin64 --- works fine. maybe you should add something to the script to install and activate latest nightly in one step.
Re: What could this OptLink Error mean?
On Saturday, 30 December 2017 at 00:49:48 UTC, user1234 wrote: The deps have to be rebuild too. After downloading dmd 78, it started to work. It's likely you were right about the issue, DUB rebuilt everything after detecting new compiler version. Sorry for my late reply.
Re: Passing a type as paramter to super
On 1/4/18 2:19 PM, Marc wrote:
On Thursday, 4 January 2018 at 19:16:03 UTC, Marc wrote:
For code generation purposes, I'd like to pass a type name to base
class. I'm not sure if it's supported, I didn't find anything at
documentation for class constructor but it does compile:
class A {
static {
int a, b;
}
this(T)() {
}
}
then do something like this:
class B {
this() {
super!B;
}
}
but I got the error:
found ! when expecting ; following statement
sorry I mean define class b as following:
class B : A {
this() {
super!B;
}
}
AFAIK, you can't explicitly use template parameters for constructors, it
has to be IFTI. Ali has the right idea for capturing the type of this.
But it only goes one level deep, it doesn't give you the full derived
type if there are 2 or more levels.
Note, you may want to simply define a separate non-constructor for this
purpose, where you have full control.
-Steve
Re: Passing a type as paramter to super
On 01/04/2018 11:16 AM, Marc wrote:
For code generation purposes, I'd like to pass a type name to base
class. I'm not sure if it's supported, I didn't find anything at
documentation for class constructor but it does compile:
class A {
static {
int a, b;
}
this(T)() {
}
}
then do something like this:
class B {
this() {
super!B;
}
}
but I got the error:
found ! when expecting ; following statement
Checkout the "this template parameter":
class A {
static {
int a, b;
}
this(this T)() {
import std.stdio;
writeln(T.stringof);
}
}
class B : A {
this() {
super();
}
}
void main() {
auto b = new B();
}
Ali
Re: How do i activate the latest nightly without knowing the date ?
On Thursday, 4 January 2018 at 20:28:47 UTC, user789 wrote: After using the script to setup DMD nightly i read in the output that i have to run "source ~/dlang/dmd-master-2018-01-04/activate" to have DMD DUB etc working. Is there another activation command, working without the date, suitable for a cron job ? "source ~/dlang/dmd-nightly/activate" doesn't work but would be better when you just want to use the nightly
How do i activate the latest nightly without knowing the date ?
After using the script to setup DMD nightly i read in the output that i have to run "source ~/dlang/dmd-master-2018-01-04/activate" to have DMD DUB etc working. Is there another activation command, working without the date, suitable for a cron job ?
Re: Help optimizing UnCompress for gzipped files
On 1/4/18 1:57 PM, Christian Köstlin wrote: Thanks Steve, this runs now faster, I will update the table. Still a bit irked that I can't match the C speed :/ But, I can't get your C speed to duplicate on my mac even with gcc, so I'm not sure where to start. I find it interesting that you are not using any optimization flags for gcc. -Steve
Re: Help optimizing UnCompress for gzipped files
On 1/4/18 1:57 PM, Christian Köstlin wrote: Sorry, but I do not get how to cleanup the mallocated memory :) Could you help me out here? free(mypipe.window.ptr); At the moment this works because you haven't released any data from the front. But eventually, I will need to figure out how to handle this when it's not so neat. -Steve
Re: how do I get only static member of a class?
On Thursday, 4 January 2018 at 00:02:24 UTC, Ali Çehreli wrote:
On 01/03/2018 03:48 PM, Marc wrote:
I found no way with __traits() on std.traits. I found
isStaticFunction and isStaticArray but nothing about a member.
Is this by desgin?
Give a class like:
class C { static int a, b, c; int d; }
I'd like to get a, b and c.
I'm using this:
__traits(allMembers, C)
class C { static int a, b, c; int d; }
string[] staticMembers(T)() {
string[] statics;
foreach (m; __traits(derivedMembers, T)) {
import std.traits : hasStaticMember;
static if (hasStaticMember!(T, m)) {
statics ~= m;
}
}
return statics;
}
void main() {
pragma(msg, staticMembers!C);
}
Ali
Thanks again, Ali
Re: Dub and libraries
On Thursday, January 04, 2018 19:05:59 Russel Winder via Digitalmars-d-learn wrote: > Is it the case that, for library things on the Dub repository, Dub will > only create library archives, .a, that it is unable to create shared > objects and DLLs? Well, you can configure a dub project to be a shared library by making its target type "dynamicLibrary", in which case, that's all it's going to build as, so something would be very wrong if it didn't build that way when using code.dlang.org. However, I expect that most folks simply use "library" for the target type and not "dynamicLibrary", and in that case, I believe that it does static by default, and I don't know how you can control that. If you have more detailed questions, I'd suggest asking on the forum intended for dub: http://forum.rejectedsoftware.com/ - Jonathan M Davis
Passing a type as paramter to super
For code generation purposes, I'd like to pass a type name to
base class. I'm not sure if it's supported, I didn't find
anything at documentation for class constructor but it does
compile:
class A {
static {
int a, b;
}
this(T)() {
}
}
then do something like this:
class B {
this() {
super!B;
}
}
but I got the error:
found ! when expecting ; following statement
Re: Passing a type as paramter to super
On Thursday, 4 January 2018 at 19:16:03 UTC, Marc wrote:
For code generation purposes, I'd like to pass a type name to
base class. I'm not sure if it's supported, I didn't find
anything at documentation for class constructor but it does
compile:
class A {
static {
int a, b;
}
this(T)() {
}
}
then do something like this:
class B {
this() {
super!B;
}
}
but I got the error:
found ! when expecting ; following statement
sorry I mean define class b as following:
class B : A {
this() {
super!B;
}
}
Dub and libraries
Is it the case that, for library things on the Dub repository, Dub will only create library archives, .a, that it is unable to create shared objects and DLLs? -- Russel. == Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Roadm: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk signature.asc Description: This is a digitally signed message part
Re: Help optimizing UnCompress for gzipped files
On 04.01.18 16:53, Steven Schveighoffer wrote:
> On 1/3/18 3:28 PM, Steven Schveighoffer wrote:
>
>> Stay tuned, there will be updates to iopipe to hopefully make it as
>> fast in this microbenchmark as the C version :)
>
> v0.0.3 has been released. To take advantage of using malloc/realloc, you
> can use std.experimental.allocator.mallocator.Mallocator as the
> BufferManager's allocator.
>
> e.g.:
>
> auto myPipe = openDev("file.gz").bufd // not here
> .unzip!Mallocator; // here
>
> myPipe.ensureElems(); // this works now.
>
> auto bytesRead = myPipe.window.length;
>
> The reason you don't need it on the first bufd is because that is the
> buffer of the file stream, which isn't going to grow.
>
> Note: the buffer manager does not free the data on destruction! You are
> responsible for freeing it (if you use Mallocator).
Thanks Steve,
this runs now faster, I will update the table.
Sorry, but I do not get how to cleanup the mallocated memory :)
Could you help me out here?
--
Christian Köstlin
Re: Error: variable i cannot be read at compile time
On 01/04/2018 08:51 AM, Vino wrote:
> auto read () {
[...]
> return tuple(Ucol1, Ucol2, Ucol3, rSize);
> }
read() returns a tuple of values of different types.
> for(int i = 0; i < Size; i++) {
> typeof(read()[i]) Datacol;
typeof is a compile-time expression but there cannot be a consistent
result to that expression when i is not known at compile-time.
You might try using a 'static foreach' but this time Size is not a
compile-time expression:
static foreach(i; 0 .. Size) {
typeof(read()[i]) Datacol;
Error: variable Size cannot be read at compile time
Ali
Re: Error: variable foo conflicts with struct foo
On Thursday, 4 January 2018 at 17:59:55 UTC, Colin wrote: How can the compiler know which symbol is which symbol if everything has the same name? Standard practice is to capitalise type names and camelCase variable names. The C# compiler has no trouble understanding code like that, so I was wondering if it is a design decision for the language, or simply something not yet supported by the compiler. I couldn't find any documentation on it either. Perhaps someone can point me to it? I did manage to work around the issue by fully qualifying the type: private __gshared kernel.idt.IDT_register IDT_register; Also, I'm well aware that I'm not following the standard practices regarding naming, but I'm just toying with -betterC so I'm not really concerned about it.
Re: Error: variable foo conflicts with struct foo
On Thursday, 4 January 2018 at 17:45:35 UTC, Stijn wrote:
Why is it not allowed for a variable name to match a type name?
The following example fails with "Error: variable foo conflicts
with struct foo"
struct foo {}
foo foo;
Imagine if you then tried to write something like
alias bar = foo;
Or, imagine you placed the above declarations into the module
`mymod.d`, and then in a different module wrote
import mymod: foo;
Which `foo` should these refer to, the type or the variable?
Re: Error: variable foo conflicts with struct foo
On Thursday, 4 January 2018 at 17:45:35 UTC, Stijn wrote:
Why is it not allowed for a variable name to match a type name?
The following example fails with "Error: variable foo conflicts
with struct foo"
struct foo {}
foo foo;
How can the compiler know which symbol is which symbol if
everything has the same name?
Standard practice is to capitalise type names and camelCase
variable names.
Error: variable foo conflicts with struct foo
Why is it not allowed for a variable name to match a type name?
The following example fails with "Error: variable foo conflicts
with struct foo"
struct foo {}
foo foo;
Re: Using iopipe to stream a gzipped file
On Thursday, 4 January 2018 at 15:48:21 UTC, Steven Schveighoffer wrote: It's now been updated, see version 0.0.3. Note, the performance isn't something I focused on. I'll note that gzcat | wc -l is 2x faster than your simple example on that file. I can think of a couple reasons for this: 1. gzcat may use mmap to increase read speed 2. gzcat may read larger chunks at once (this can be tuned using iopipe as well, just set the optimizedReadSize). 3. gzcat file.gz | iopipe_byline -nooutput is about 20% faster than using wc -l, so it's definitely not the line parsing. Let me know if this works correctly for your other test cases! If not, file an issue: https://github.com/schveiguy/iopipe/issues -Steve That works perfectly, thank you very much!
Error: variable i cannot be read at compile time
Hi All,
Request your help on the below error for the below program.
Error:
CReadCol.d(20): Error: variable i cannot be read at compile time
CReadCol.d(21): Error: variable i cannot be read at compile time
CReadCol.d(22): Error: variable i cannot be read at compile time
Program
import std.algorithm: joiner, sort, countUntil, uniq;
import std.container.array;
import std.csv: csvReader;
import std.stdio: File, writeln;
import std.typecons: Tuple, tuple;
auto read () {
Array!string Ucol1, Ucol2;
Array!int Ucol3;
int rSize;
auto file =
File("C:\\Users\\bheev1\\Desktop\\Current\\Script\\Others\\ColRead.csv", "r");
foreach (record;
file.byLineCopy.joiner("\n").csvReader!(Tuple!(string, string,
int)))
{ Ucol1.insertBack(record[0]); Ucol2.insertBack(record[1]);
Ucol3.insertBack(record[2]); rSize = record.length; }
return tuple(Ucol1, Ucol2, Ucol3, rSize);
}
/***/
auto Master (int Size) {
Array!int Keycol;
for(int i = 0; i < Size; i++) {
typeof(read()[i]) Datacol;
Datacol.insertBack(sort(read[i].dup[]).uniq);
foreach(k; read[i]) { Keycol.insertBack(Datacol[].countUntil(k));
} }
return tuple (Datacol[], Keycol[]);
}
void main () {
int Size = read[3];
writeln(Master(Size));
}
From,
Vino.B
Re: Single type of a tuple return type function
On Thursday, 4 January 2018 at 16:09:07 UTC, Simen Kjærås wrote: On Thursday, 4 January 2018 at 15:50:35 UTC, Vino wrote: [...] ReturnType!Fn[0] tries to give you the 0th field of the tuple, but as the error message indicates, you can't do that without an instance. What you want is the *type* of the field, as given by typeof: typeof(ReturnType!Fn[0]) Dcol; This can be made a bit simpler by noticing that ReturnType is unnecessary here: typeof(Fn()[0]) Dcol; However, if Fn() takes a bunch of complex parameters, this might not actually be simpler. -- Simen HI Simen, Thank you very much, your solution was helpful. From, Vino.B
Re: Single type of a tuple return type function
On Thursday, 4 January 2018 at 15:50:35 UTC, Vino wrote:
Hi All,
Request your help, on how o find the single type of a tuple
return type function, eg,
auto Fn (){
Array!string a;
Array!int b;
Array!ulong c;
return tuple(a, b, c);
}
if we use "ReturnType!Fn" it gives us the output as
(Array!string,Array!int, Array!ulong) but what is need is the
return type of each of the value as
a = Array!string; b = Array!int; c = Array!ulong
void main () {
ReturnType!Fn[0] Dcol; //similar like this line
writeln(Dcol[]);
}
From,
Vino.B
ReturnType!Fn[0] tries to give you the 0th field of the tuple,
but as the error message indicates, you can't do that without an
instance. What you want is the *type* of the field, as given by
typeof:
typeof(ReturnType!Fn[0]) Dcol;
This can be made a bit simpler by noticing that ReturnType is
unnecessary here:
typeof(Fn()[0]) Dcol;
However, if Fn() takes a bunch of complex parameters, this might
not actually be simpler.
--
Simen
Single type of a tuple return type function
Hi All,
Request your help, on how o find the single type of a tuple
return type function, eg,
auto Fn (){
Array!string a;
Array!int b;
Array!ulong c;
return tuple(a, b, c);
}
if we use "ReturnType!Fn" it gives us the output as
(Array!string,Array!int, Array!ulong) but what is need is the
return type of each of the value as
a = Array!string; b = Array!int; c = Array!ulong
void main () {
ReturnType!Fn[0] Dcol; //similar like this line
writeln(Dcol[]);
}
From,
Vino.B
Re: Help optimizing UnCompress for gzipped files
On 1/3/18 3:28 PM, Steven Schveighoffer wrote:
Stay tuned, there will be updates to iopipe to hopefully make it as fast
in this microbenchmark as the C version :)
v0.0.3 has been released. To take advantage of using malloc/realloc, you
can use std.experimental.allocator.mallocator.Mallocator as the
BufferManager's allocator.
e.g.:
auto myPipe = openDev("file.gz").bufd // not here
.unzip!Mallocator; // here
myPipe.ensureElems(); // this works now.
auto bytesRead = myPipe.window.length;
The reason you don't need it on the first bufd is because that is the
buffer of the file stream, which isn't going to grow.
Note: the buffer manager does not free the data on destruction! You are
responsible for freeing it (if you use Mallocator).
-Steve
Re: Using iopipe to stream a gzipped file
On 1/4/18 7:23 AM, Andrew wrote: On Thursday, 4 January 2018 at 12:15:27 UTC, Steven Schveighoffer wrote: In any case, I'll figure out how to deal with concatenated gzip file, and update iopipe. Next version will focus on a bunch of stuff relating to the 2 zip threads recently posted here. That would be really great for me, thank you! It's now been updated, see version 0.0.3. Note, the performance isn't something I focused on. I'll note that gzcat | wc -l is 2x faster than your simple example on that file. I can think of a couple reasons for this: 1. gzcat may use mmap to increase read speed 2. gzcat may read larger chunks at once (this can be tuned using iopipe as well, just set the optimizedReadSize). 3. gzcat file.gz | iopipe_byline -nooutput is about 20% faster than using wc -l, so it's definitely not the line parsing. Let me know if this works correctly for your other test cases! If not, file an issue: https://github.com/schveiguy/iopipe/issues -Steve
Re: How do you do "const nazi" in D?
On Wednesday, 3 January 2018 at 17:27:38 UTC, Marc wrote:
for a safe programming, since C/C++ times I try to make thing
const as possible. locals, parameters etc anything which isn't
going to change.
How do you do that in D? immutable everywhere?
for example:
foreach(Field field; fields) {
immutable string htmlOutputfile = genHTMLFilename();
do you use immutable here?
and on:
Field fromFile(in string filename) {
do you use in here if filename isn't going to change?
I'm trying to make my own const nazi guide for D, if anyone
would like to give your opinion on this, I would be glad.
You ma find Ali Cehreli's talk from DConf 2013 [1] useful. It
includes some guidelines (basically recommendations) on how to
use immutable and const.
[1] https://www.youtube.com/watch?v=mPr2UspS0fE
Re: How do you do "const nazi" in D?
On 1/3/18 12:27 PM, Marc wrote:
for a safe programming, since C/C++ times I try to make thing const as
possible. locals, parameters etc anything which isn't going to change.
How do you do that in D? immutable everywhere?
For parameters, I'd recommend const, not immutable, as const allows more
to be passed in. If you need to return something from the parameters,
inout should be what you use.
for example:
foreach(Field field; fields) {
immutable string htmlOutputfile = genHTMLFilename();
immutable is OK here, as it's not a parameter. Note, you can drop the
"string" type, as it will be inferred.
A further note here -- immutable string can convert implicitly to
mutable string, because the data underneath is also immutable. But you
won't be so lucky with other data types.
You may want to be cautious when declaring something const/immutable
inside your functions, as it may not convert for calls to other
libraries that haven't been properly "const nazi'd".
and on:
Field fromFile(in string filename) {
do you use in here if filename isn't going to change?
It depends completely on your internals. In this case, you are accepting
a string by value. The string contents are referenced, but those are
already immutable. You can't change anything the caller passed in, even
if this was not tagged 'in'. All you can do is rebind filename locally
to something else. If it were me, I would do this:
Field fromFile(const(char)[] filename)
Or potentially this if you never are going to change filename internally:
Field fromFile(in char[] filename)
This allows passing any flavor of char array, so it provides a wider
amount of access.
I'm trying to make my own const nazi guide for D, if anyone would like
to give your opinion on this, I would be glad.
If you send me email when you are ready, I'll review!
As Jonathan implied, const is viral. If you try using proper const code
with libraries that don't use it, you will have lots of trouble. So it's
not always worth it.
-Steve
Re: Using iopipe to stream a gzipped file
On Thursday, 4 January 2018 at 12:15:27 UTC, Steven Schveighoffer wrote: On 1/4/18 7:01 AM, Andrew wrote: Ah thank you, that makes sense. These types of files are compressed using the bgzip utility so that the file can be indexed meaning specific rows extracted quickly (there's more details of this here http://www.htslib.org/doc/tabix.html and the code can be found here: https://github.com/samtools/htslib/blob/develop/bgzf.c) Hm... that utility seems to say it will result in bgz file extension? So this must be an extraction from one of those files? In any case, I'll figure out how to deal with concatenated gzip file, and update iopipe. Next version will focus on a bunch of stuff relating to the 2 zip threads recently posted here. Thanks! -Steve That would be really great for me, thank you! By default bgzip produces a file with the standard .gz extension. Looking at the code it adds an extra field to the standard gzip header: /* BGZF/GZIP header (speciallized from RFC 1952; little endian): +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+ | 31|139| 8| 4| 0| 0|255| 6| 66| 67| 2|BLK_LEN| +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+ BGZF extension: ^ ^ ^ ^ | | | | FLG.EXTRA XLEN B C BGZF format is compatible with GZIP. It limits the size of each compressed block to 2^16 bytes and adds and an extra "BC" field in the gzip header which records the size. */ Thanks again! Andrew
Re: Using iopipe to stream a gzipped file
On 1/4/18 7:01 AM, Andrew wrote: Ah thank you, that makes sense. These types of files are compressed using the bgzip utility so that the file can be indexed meaning specific rows extracted quickly (there's more details of this here http://www.htslib.org/doc/tabix.html and the code can be found here: https://github.com/samtools/htslib/blob/develop/bgzf.c) Hm... that utility seems to say it will result in bgz file extension? So this must be an extraction from one of those files? In any case, I'll figure out how to deal with concatenated gzip file, and update iopipe. Next version will focus on a bunch of stuff relating to the 2 zip threads recently posted here. Thanks! -Steve
Re: Gc/D_runtime prevents dangling pointers?
On 1/4/18 6:50 AM, thedeemon wrote: On Thursday, 4 January 2018 at 11:05:25 UTC, tipdbmp wrote: It seems that the '~=' operator "knows" that there's a reference to 'a's old memory and it keeps it around instead of freeing it. It's just not its job to free that memory. That memory is freed later by GC, when it's safe to do so. Yeah, to further clarify, the array runtime has no idea who is looking at the memory. It doesn't "know" about references, it just relies on the GC to clean up the garbage if it actually is garbage. -Steve
Re: Using iopipe to stream a gzipped file
On Thursday, 4 January 2018 at 02:44:09 UTC, Steven Schveighoffer wrote: On 1/3/18 12:03 PM, Andrew wrote: Thanks for looking into this. So it looks like the file you have is a concatenated gzip file. If I gunzip the file and recompress it, it works properly. Looking at the docs of zlib inflate [1]: " Unlike the gunzip utility and gzread() ..., inflate() will not automatically decode concatenated gzip streams. inflate() will return Z_STREAM_END at the end of the gzip stream. The state would need to be reset to continue decoding a subsequent gzip stream." So what is happening is the inflate function is returning Z_STREAM_END, and I'm considering the stream done from that return code. I'm not sure yet how to fix this. I suppose I can check if any more data exists, and then re-init and continue. I have to look up what a concatenated gzip file is. gzread isn't good for generic purposes, because it requires an actual file input (I want to support any input type, including memory data). -Steve [1] https://github.com/dlang/phobos/blob/master/etc/c/zlib.d#L874 Ah thank you, that makes sense. These types of files are compressed using the bgzip utility so that the file can be indexed meaning specific rows extracted quickly (there's more details of this here http://www.htslib.org/doc/tabix.html and the code can be found here: https://github.com/samtools/htslib/blob/develop/bgzf.c)
Re: Gc/D_runtime prevents dangling pointers?
On Thursday, 4 January 2018 at 11:05:25 UTC, tipdbmp wrote: What is your definition of a dangling pointer? A pointer pointing to freed memory, which presumably '[0]' should be because it reallocates. It allocates a larger array, but the old version is not freed up front. Right because there might be live references to the old data. Your pointer is such live reference. So while this pointer references that old version of the array, it's not freed. Only after no references are left the GC will make it free (and only when next GC cycle happens, not immediately). It seems that the '~=' operator "knows" that there's a reference to 'a's old memory and it keeps it around instead of freeing it. It's just not its job to free that memory. That memory is freed later by GC, when it's safe to do so.
Re: Help optimizing UnCompress for gzipped files
On 1/4/18 4:47 AM, Christian Köstlin wrote:
I added now a c variant, that does always malloc/memcpy/free. Its much
slower for sure.
Also I put some output in thats shows when a real realloc happens. Its
like you said:
did a real realloc
did a real realloc
did a real realloc
did a real realloc
did a real realloc
did a real realloc
did not a real realloc
did not a real realloc
did not a real realloc
did not a real realloc
did not a real realloc
did not a real realloc
did not a real realloc
did not a real realloc
did a real realloc
did not a real realloc
funny thing is, i do not always get the same sequence of real reallocs.
Another thing I did was print out the size of the buffer before the
realloc. In other words:
auto oldsize = buffer.length;
auto oldptr = buffer.ptr;
buffer = (cast(ubyte *)realloc(buffer.ptr, newsize))[0 .. newsize];
if(buffer.ptr != oldptr) writeln("moved: ", oldsize);
Even the sizes malloc chooses for its big leap are not consistent from
run to run!
I'm wondering if possibly when you keep reallocing a large block, and it
doesn't have enough memory, needs more from the system, it just tacks it
onto the end of the existing big block. This would make the performance
for this microbenchmark highly dependent on not doing any other allocations!
BTW, my numbers are consistent, but there is a difference from yours.
For example, my C runs are about 430ms, and my fast D runs are about
650-700ms. I never get the 200ms range runs that you are seeing. I have
a similar machine, but still running Sierra.
-Steve
Re: Gc/D_runtime prevents dangling pointers?
What is your definition of a dangling pointer? A pointer pointing to freed memory, which presumably '[0]' should be because it reallocates. It seems that the '~=' operator "knows" that there's a reference to 'a's old memory and it keeps it around instead of freeing it. I just don't understand the mechanism behind this.
Re: how to localize console and GUI apps in Windows
On Friday, 29 December 2017 at 18:13:04 UTC, H. S. Teoh wrote: If the problem is in readln(), then you probably need to read the input in binary (i.e., as ubyte[]) and convert it manually. Could you kindly explain how I can read console input into binary ubyte[]?
Re: how to localize console and GUI apps in Windows
On Friday, 29 December 2017 at 18:13:04 UTC, H. S. Teoh wrote: On Fri, Dec 29, 2017 at 10:35:53AM +, Andrei via Digitalmars-d-learn wrote: This may be endurable if you write an application where Russian is only one of rare options, and what if your whole environment is totally Russian? You mean if your environment uses a non-UTF encoding? If your environment uses UTF, there is no problem. I have code with strings in Russian (and other languages) embedded, and it's no problem because everything is in Unicode, all input and all output. No, I mean difficulties to write a program based on non-ASCII locales. Every programming language learning since C starts with a "hello world" program which every non-English programmer essentially tries to translate to native language - and gets unreadable mess on the screen. Thousands try, hundreds look for a solution, dozens find it, and a few continue with the new language. That's not because these programmers cannot read English text-books, they can. That's because they want to write non-English programs for non-English people, and that's essential. And there are many programming languages (or rather their runtimes) which do not suffer such a deficiency. That's the reason for UNICODE adoption all over the programming world - including D language, but what's the good for me if I can write in a D program a UTF8 string with my native language text, and get the same unreadable mess on the screen? Yes, a new language in development can lack support for some features, but this forum branch shows that a simple and handy solution exists - yet nobody cares to bring it to the first pages of every text-book for beginners, at least as a footnote. Thus thousands of potential new language fans are lost from start. But I understand that in Windows you may not have this luxury. So you have to deal with codepages and what-not. Converting back and forth is not a big problem, and it actually also solves the problem of string comparisons, because std.uni provides utilities for collating strings, etc.. But it only works for Unicode, so you have to convert to Unicode internally anyway. Also, for static strings, it's not hard to make the codepage mapping functions CTFE-able, so you can actually write string literals in a codepage and have the compiler automatically convert it to UTF-8. The other approach, if you don't like the idea of converting codepages all the time, is to explicitly work in ubyte[] for all strings. Or, preferably, create your own string type with ubyte[] representation underneath, and implement your own comparison functions, etc., then use this type for all strings. Better yet, contribute this to code.dlang.org so that others who have the same problem can reuse your code instead of needing to write their own. I'd definitely try this if I decide to use D language for my purposes (which not settled yet). But to decide I need some experience, and for now it stopped at reading the user's input (for training I intend to translate into D my recent rather complex interactive C# program). Still this does not decide localized input problem: any localized input throws an exception “std.utf.UTFException... Invalid UTF-8 sequence”. Is the exception thrown in readln() or in writeln()? If it's in writeln(), it shouldn't be a big deal, you just have to pass the data returned by readln() to fromKOI8 (or whatever other codepage you're using). If the problem is in readln(), then you probably need to read the input in binary (i.e., as ubyte[]) and convert it manually. Unfortunately, there's no other way around this if you're forced to use codepages. The ideal situation is if you can just use Unicode throughout your environment. But of course, sometimes you have no choice. It depends. If I avoid proper console code page initializing, I see in debugger that runtime reads the user's input as CP866 (MS DOS) Cyrillic and then throws the exception "Invalid UTF-8 sequence" when trying to handle it as UTF8 string (in particular by strip() or writeln() functions). This situation seems quite manageable by code page conversions you've mentioned above. I've tried first library function found (std.windows.charset), and got a rather fanciful working statement: response = fromMBSz((readln()~"\0").ptr, 1).strip(); which assigns correct Latin/Cyrillic contents to the response variable. And if I initialize console with SetConsoleCP(65001) statement things get worse, as I've said above. Then readln() statement returns an empty string and something gets broken inside the runtime, because any further readln() statements do not wait for user input, and return empty strings immediately.
Re: Help optimizing UnCompress for gzipped files
I added now a c variant, that does always malloc/memcpy/free. Its much slower for sure. Also I put some output in thats shows when a real realloc happens. Its like you said: did a real realloc did a real realloc did a real realloc did a real realloc did a real realloc did a real realloc did not a real realloc did not a real realloc did not a real realloc did not a real realloc did not a real realloc did not a real realloc did not a real realloc did not a real realloc did a real realloc did not a real realloc funny thing is, i do not always get the same sequence of real reallocs. -- Christian Köstlin
Re: No modification of pointer values in safe functions?
On Wednesday, 3 January 2018 at 23:49:33 UTC, H. S. Teoh wrote:
On Wed, Jan 03, 2018 at 03:42:07PM -0700, Jonathan M Davis via
Digitalmars-d-learn wrote:
On Wednesday, January 03, 2018 22:25:16 Mark via
Digitalmars-d-learn wrote:
[...]
> https://dlang.org/spec/function.html#safe-functions
>
> "The following operations are not allowed in safe functions:
>
> [...]
> - No modification of pointer values.
> [...]"
Then the spec needs to be clarified. Assignment is fine but
other types of mutation are not.
Here's my attempt to clarify it:
https://github.com/dlang/dlang.org/pull/2050
T
Okay, that's clear now.
Ali's book wasn't clear on that either ("Pointers cannot be
mutated"). I'll ask him to clarify it.
