Re: Class instance alignment

[tsbockman via Digitalmars-d-learn]

On Tuesday, 23 February 2021 at 03:53:00 UTC, tsbockman wrote:
size_t alignedSize(size_t typeSize, size_t typeAlignment) pure 
@safe nothrow @nogc {

version(assert) {
import core.bitop : bsr;
assert(typeAlignment == (size_t(1) << 
bsr(typeAlignment)));

}
size_t ret = typeSize & ~(typeAlignment - 1);
ret += (ret < typeSize)? typeAlignment : 0;
return ret;
}


Better:

size_t alignedSize(size_t typeSize, size_t typeAlignment) pure 
@safe nothrow @nogc {

version(assert) {
import core.bitop : bsr;
assert(typeAlignment == (size_t(1) << 
bsr(typeAlignment)));

}
const alignMask = typeAlignment - 1;
return (typeSize + alignMask) & ~alignMask;
}

Re: Problem Computing Dot Product with mir

[9il via Digitalmars-d-learn]

On Tuesday, 23 February 2021 at 03:48:15 UTC, Max Haughton wrote:

On Monday, 22 February 2021 at 07:14:26 UTC, 9il wrote:
On Sunday, 21 February 2021 at 16:18:05 UTC, Kyle Ingraham 
wrote:
I am trying to convert sRGB pixel values to XYZ with mir 
using the following guide: 
http://www.brucelindbloom.com/index.html?Eqn_RGB_XYZ_Matrix.html


[...]


mir-glas is deprecated experimental project. It is worth use 
mir-blas or lubeck instead. There is also naming issue. In the 
classic BLAS naming dot refers a function that accept two 1D 
vectors.


Deprecated as in formally dead or postponed?


Postponed until someone wish to invest in BLAS library in D.

Re: Trying to reduce memory usage

[Jon Degenhardt via Digitalmars-d-learn]

On Tuesday, 23 February 2021 at 00:08:40 UTC, tsbockman wrote:
On Friday, 19 February 2021 at 00:13:19 UTC, Jon Degenhardt 
wrote:
It would be interesting to see how the performance compares to 
tsv-uniq 
(https://github.com/eBay/tsv-utils/tree/master/tsv-uniq). The 
prebuilt binaries turn on all the optimizations 
(https://github.com/eBay/tsv-utils/releases).


My program (called line-dedup below) is modestly faster than 
yours, with the gap gradually widening as files get bigger. 
Similarly, when not using a memory-mapped scratch file, my 
program is modestly less memory hungry than yours, with the gap 
gradually widening as files get bigger.


In neither case is the difference very exciting though; the 
real benefit of my algorithm is that it can process files too 
large for physical memory. It might also handle frequent hash 
collisions better, and could be upgraded to handle huge numbers 
of very short lines efficiently.


Thanks for running the comparison! I appreciate seeing how other 
implementations compare.


I'd characterize the results a differently though. Based on the 
numbers, line-dedup is materially faster than tsv-uniq, at least 
on the tests run. To your point, it may not make much practical 
difference on data sets that fit in memory. tsv-uniq is fast 
enough for most needs. But it's still a material performance 
delta. Nice job!


I agree also that the bigger pragmatic benefit is fast processing 
of files much larger than will fit in memory. There are other 
useful problems like this. One I often need is creating a random 
weighted ordering. Easy to do for data sets that fit in memory, 
but hard to do fast for data sets that do not.


--Jon

Re: Class instance alignment

[tsbockman via Digitalmars-d-learn]

On Monday, 22 February 2021 at 02:23:27 UTC, Steven Schveighoffer 
wrote:
Hm... but does TypeInfo detail alignment? If so, we can make 
this work anyway, just bump up the size needed to a power-of-2 
pool.


It doesn't even need to be a power-of-2, assuming the pools 
themselves are properly aligned - just a multiple of the 
alignment:


size_t alignedSize(size_t typeSize, size_t typeAlignment) pure 
@safe nothrow @nogc {

version(assert) {
import core.bitop : bsr;
assert(typeAlignment == (size_t(1) << 
bsr(typeAlignment)));

}
size_t ret = typeSize & ~(typeAlignment - 1);
ret += (ret < typeSize)? typeAlignment : 0;
return ret;
}

(This CTFE-able and can be memoized with a template, if desired. 
It's also just a few branchless instructions at runtime, if it's 
needed then for some reason.)

Re: Problem Computing Dot Product with mir

[Max Haughton via Digitalmars-d-learn]

On Monday, 22 February 2021 at 07:14:26 UTC, 9il wrote:
On Sunday, 21 February 2021 at 16:18:05 UTC, Kyle Ingraham 
wrote:
I am trying to convert sRGB pixel values to XYZ with mir using 
the following guide: 
http://www.brucelindbloom.com/index.html?Eqn_RGB_XYZ_Matrix.html


[...]


mir-glas is deprecated experimental project. It is worth use 
mir-blas or lubeck instead. There is also naming issue. In the 
classic BLAS naming dot refers a function that accept two 1D 
vectors.


Deprecated as in formally dead or postponed?

Re: Problem Computing Dot Product with mir

[Kyle Ingraham via Digitalmars-d-learn]

On Monday, 22 February 2021 at 07:14:26 UTC, 9il wrote:
On Sunday, 21 February 2021 at 16:18:05 UTC, Kyle Ingraham 
wrote:
I am trying to convert sRGB pixel values to XYZ with mir using 
the following guide: 
http://www.brucelindbloom.com/index.html?Eqn_RGB_XYZ_Matrix.html


[...]


mir-glas is deprecated experimental project. It is worth use 
mir-blas or lubeck instead. There is also naming issue. In the 
classic BLAS naming dot refers a function that accept two 1D 
vectors.


Your suggestion worked beautifully. I went with lubeck. I don't 
know how I missed it seeing that it is right there in the Dlang 
tour.


Thank you for writing such useful libraries.

Final code ended up being:

import std.stdio;

import kaleidic.lubeck: mtimes;
import mir.ndslice: as, byDim, fuse, map, sliced;

void main()
{
auto rgbToXyz = [0.4124564,  0.3575761,  0.1804375,
 0.2126729,  0.7151522,  0.0721750,
 0.0193339,  0.1191920,  0.9503041]
.as!double
.sliced(3, 3);

ubyte[] rgbPixels = [255, 0, 0,
 0, 255, 0,
 0, 0, 255,
 120, 120, 120];

auto xyzPixels = rgbPixels
// convert input array elements to double - lazy
.as!double
// create a slice-shell over the input array
.sliced(2, 2, 3)
// normalize pixel channels to range [0, 1]
.map!(chnl => chnl / 255.0)
// sRGB inverse compand
		.map!(chnl => chnl <= 0.04045 ? chnl / 12.92 : ((chnl + 0.055) 
/ 1.055) ^^ 2.4)

// linear RGB to XYZ
// iterator by x and y over pixels (3rd dimension)
.byDim!(0, 1) // same as .pack!1
// dot product of each pixel with conversion matrix
.map!(pixel => mtimes(rgbToXyz, pixel))
// join iterator values into a matrix
.fuse;

xyzPixels.writeln;
xyzPixels.shape.writeln;
}

Re: Trying to reduce memory usage

[tsbockman via Digitalmars-d-learn]

On Friday, 19 February 2021 at 00:13:19 UTC, Jon Degenhardt wrote:

On Wednesday, 17 February 2021 at 04:10:24 UTC, tsbockman wrote:
I spent some time experimenting with this problem, and here is 
the best solution I found, assuming that perfect 
de-duplication is required. (I'll put the code up on GitHub / 
dub if anyone wants to have a look.)


It would be interesting to see how the performance compares to 
tsv-uniq 
(https://github.com/eBay/tsv-utils/tree/master/tsv-uniq). The 
prebuilt binaries turn on all the optimizations 
(https://github.com/eBay/tsv-utils/releases).


My program (called line-dedup below) is modestly faster than 
yours, with the gap gradually widening as files get bigger. 
Similarly, when not using a memory-mapped scratch file, my 
program is modestly less memory hungry than yours, with the gap 
gradually widening as files get bigger.


In neither case is the difference very exciting though; the real 
benefit of my algorithm is that it can process files too large 
for physical memory. It might also handle frequent hash 
collisions better, and could be upgraded to handle huge numbers 
of very short lines efficiently.


Test 0 (139 MiB, 1537300 unique lines of 204):
sort source | uniq > dest
Wall time: 2.04 s
User time: 9.34 s
System time: 0.23 s
Max resident set: 385 MiB
tsv-uniq source > dest
Wall time: 0.82 s
User time: 0.73 s
System time: 0.13 s
Max resident set: 229 MiB
line-dedup source dest
Wall time: 0.58 s
User time: 0.52 s
System time 0.05 s
Max resident set: 206 MiB

Test 1 (1.4 GiB, 14338280 unique lines of 2004):
sort source | uniq > dest
Wall time: 22.9 s
User time: 107 s
System time: 2.16 s
Max resident set: 3.76 GiB
tsv-uniq source > dest
Wall time: 9.94 s
User time: 9.25 s
System time: 1.3 s
Max resident set: 2.34 GiB
line-dedup source dest
Wall time: 6.34 s
User time: 5.88 s
System time 0.44 s
Max resident set: 1.98 GiB

Test 2 (4.2 GiB, 44379959 unique lines of 6004):
sort source | uniq > dest
Wall time: 87.1 s
User time: 352.3 s
System time: 7.41 s
Max resident set: 7.84 GiB
tsv-uniq source > dest
Wall time: 42.3 s
User time: 40.7 s
System time: 4.82 s
Max resident set: 7.86 GiB
line-dedup source dest
Wall time: 23 s
User time: 19.6 s
System time 1.4 s
Max resident set: 5.96 GiB

The test files have a fractal distribution, with many lines 
having a few duplicates, and a few lines having many duplicates.

Is this a good way to do lazy evaluation?

[Jack via Digitalmars-d-learn]

I have a base class that loads a value and if it isn't net, load 
a default value. Since the value can be set, I'd like to do lazy 
evaluation of this default value until it's really needed. So 
rather have a C defValue = xxx, that makes xxx to be loaded even 
if defValue doesn't get used. So I thought I would use property 
and override it from the derived class, to archive the lazy 
evaluation effect that I'd like. Is this a good approach? are 
there better ways to do that? here's a sample code of what I'm 
talking about (I'm using int just to make it simple but the class 
in the real code is bigger and can have many instances, hence my 
attempt to optimize that):


class A
{
C defValue() { return C.a; }
C value;

void doSomething()
{
   // doSomething gets called from all derived classes
   // but value is set often, loading defValue right 
away

   // would waste time and resources
if(!value) {
value = defValue;
}
}
}

class B : A
{
// use propety to have a lazy evaluation of value
override C defValue() { return C.b; }

void foo()
{
// loadDefvalue, if value not set, lazily...
doSomething();
// do something else...
}
}

class C
{
int n;

this(int v)
{
n = v;
}

static this()
{
a = new C(10);
b = new C(11);
}

void load()
{
// do something with n
}

static
{
C a;
C b;
 }
}

Re: Is this the proper way to do it?

[Jack via Digitalmars-d-learn]

On Saturday, 13 February 2021 at 07:08:58 UTC, mw wrote:

On Saturday, 13 February 2021 at 05:52:34 UTC, Jack wrote:
I have a base class A, where I make specific operator 
depending on the derived class type. Currently I'm using 
something like this:


c is a class derived from A
bool shouldDoX = (cast(X)c) !is null || (cast(Y)c) !is null || 
(cast(K)c) !is null ... ;


as the number of cast(C) !is null is growing, I'm afraid of 
this being a inelegant or even poor performance approach. How 
would you do that?


Isn't that what virtual function is designed for?

```
class Base {
  bool shouldDoX() {return false;}
}

class Derived: Base {
  bool shouldDoX() {return true;}
}

class Derived2: Derived {
  bool shouldDoX() {return false;}
}

...

```


sounds a better approach, I ended up using this. Lots of cast(X), 
cast(Y), etc is probably slow and gets messy with time.

Re: Is this the proper way to do it?

[Jack via Digitalmars-d-learn]

On Saturday, 13 February 2021 at 19:40:43 UTC, frame wrote:

On Saturday, 13 February 2021 at 05:52:34 UTC, Jack wrote:
I have a base class A, where I make specific operator 
depending on the derived class type. Currently I'm using 
something like this:


c is a class derived from A
bool shouldDoX = (cast(X)c) !is null || (cast(Y)c) !is null || 
(cast(K)c) !is null ... ;


as the number of cast(C) !is null is growing, I'm afraid of 
this being a inelegant or even poor performance approach. How 
would you do that?


I would just use an (empty) interface on that classes and do 
test for that.


i did consider that too but ended up with virtual functions

Re: Is this the proper way to do it?

[Jack via Digitalmars-d-learn]

On Saturday, 13 February 2021 at 09:54:28 UTC, Rumbu wrote:

On Saturday, 13 February 2021 at 05:52:34 UTC, Jack wrote:
I have a base class A, where I make specific operator 
depending on the derived class type. Currently I'm using 
something like this:


c is a class derived from A
bool shouldDoX = (cast(X)c) !is null || (cast(Y)c) !is null || 
(cast(K)c) !is null ... ;


as the number of cast(C) !is null is growing, I'm afraid of 
this being a inelegant or even poor performance approach. How 
would you do that?


Option 1, reverse the condition, && op will shortcut boolean 
conditions


bool shouldNotDoX = cast(X)c && cast(Y)c && cast(K)c && ...

Option 2, reverse the condition by testing the classes that are 
not supposed to "do" it, if you have less classes in that 
category.


bool shoudldNotDoX = cast(P)c || cast(Q)c


I ended up using virtual functions, this cast and conditions 
would get too big, ugly and messy with time

Re: how to make this function nothrow?

[Jack via Digitalmars-d-learn]

On Tuesday, 16 February 2021 at 00:39:33 UTC, Steven 
Schveighoffer wrote:

On 2/15/21 4:04 PM, Jack wrote:
I have to make my function nothrow because the function that 
calls it (not written by me) is nothrow. So I need to wrap my 
code in a try-catch() but how will I report the error message, 
if the toString() from Throwable isn't nothrow? how do I get 
out this circular dependence?



void f() nothrow
{
   import std.conv : to;

     try
     {
     // do something
     }
     catch(Throwable th)
     {
     auto err = th.toString;
     }
}

I can't use err variable, it result in error:

function object.Throwable.toString is not nothrow

obviously, insert a try-catch() within catch() is a circular 
dependence and doesn't solve the problem either (even if it, I 
think it would be quite ugly)


https://dlang.org/phobos/std_exception.html#assumeWontThrow

import std.exception;

auto err = assumeWontThrow(th.toString, "oops, toString threw 
something!");


-Steve


I didn't know about that function, I'll be using this one from 
now. Thanks!

Adam and Ali thank you guys too, helpful always

Re: How to get Laptop battery level

[H. S. Teoh via Digitalmars-d-learn]

On Mon, Feb 22, 2021 at 08:59:01PM +, Greatsam4sure via Digitalmars-d-learn 
wrote:
> Dlang is a system programming language. How do I access the battery
> level of my system using code in dlang? I will appreciate vide sample

There is no universal API for this.  It depends on which OS you're
using.  For that, you need to consult your OS's documentation. Most
likely there will be some kind of C API or system library that could
give you this information.

D can't possibly provide a universal API for something like this because
it varies too widely -- some platforms do not have any meaningful
notion of "battery level", whereas other systems may well have multiple
"battery levels" and it would be ambiguous which one is meant.  If you
want to do systems programming you have to know the system you're
programming, and be prepared to call OS APIs yourself.


T

-- 
The irony is that Bill Gates claims to be making a stable operating system and 
Linus Torvalds claims to be trying to take over the world. -- Anonymous

Re: How to get Laptop battery level

[Dukc via Digitalmars-d-learn]

On Monday, 22 February 2021 at 21:14:48 UTC, Dukc wrote:
Note that DGame seems to be currently unmaintained - it might 
have some bits that do not compile anymore. SDL2 is a commonly 
used library though - you should be able to find more examples 
about it if you need to.


Plus, the example I linked is not really the simple minimalistic 
one you want if battery level is all you want to figure out. 
Probably the derelict-sdl2 front page is what you're looking for 
that instead:


https://code.dlang.org/packages/derelict-sdl2

Re: How to get Laptop battery level

[Dukc via Digitalmars-d-learn]

On Monday, 22 February 2021 at 20:59:01 UTC, Greatsam4sure wrote:
Dlang is a system programming language. How do I access the 
battery level of my system using code in dlang?


As a systems language, D could access the hardware interface 
directly, if you knew it and loaded your program as a kernel 
driver - if your operating system lets you do that.


Now, I don't think that was the most practical way to approach 
the problem :-). Instead, I recommend you use the SDL2 multimedia 
library to give you a more portable interface. Usage example in D 
here: 
https://github.com/Dgame/Dgame/blob/master/source/Dgame/System/Battery.d


Note that the D module I linked does not contain the machinery to 
initialize SDL2. This module seems to contain that:

https://github.com/Dgame/Dgame/blob/master/source/Dgame/Window/Window.d

Note that DGame seems to be currently unmaintained - it might 
have some bits that do not compile anymore. SDL2 is a commonly 
used library though - you should be able to find more examples 
about it if you need to.

Re: How to get Laptop battery level

[Greatsam4sure via Digitalmars-d-learn]

On Monday, 22 February 2021 at 20:59:01 UTC, Greatsam4sure wrote:
Dlang is a system programming language. How do I access the 
battery level of my system using code in dlang? I will 
appreciate code sample


Sorry I mean to say code sample

How to get Laptop battery level

[Greatsam4sure via Digitalmars-d-learn]

Dlang is a system programming language. How do I access the 
battery level of my system using code in dlang? I will appreciate 
vide sample

Re: How to get output of piped process?

[frame via Digitalmars-d-learn]

On Monday, 22 February 2021 at 13:23:40 UTC, Danny Arends wrote:


https://github.com/DannyArends/DaNode/blob/master/danode/process.d

Danny


This example shows how easy it is to implement a non-blocking 
stream. Phobos knows this for sockets but not for pipes?

Re: How to get output of piped process?

[Danny Arends via Digitalmars-d-learn]

On Friday, 19 February 2021 at 15:39:25 UTC, kdevel wrote:
On Friday, 19 February 2021 at 13:42:46 UTC, Steven 
Schveighoffer wrote:


[...]


[...]


Sure.


[...]


As application programmer I don't want to check any error codes.
Thankfully I don't have to in D. There is a nice off-topic 
example

from the linux kernel [1] what happens when people do not check
return values [1].


[...]


Fortunately the D runtime /does/ take care and it throws---if 
the signal

is ignored beforehand. I filed issue 21649.

[1] 
move RLIMIT_NPROC check from set_user() to 
do_execve_common()


Perhaps a bit late, but this is how I deal with pipes and 
spawnShell.

Read one byte at a time from stdout and stderr:

https://github.com/DannyArends/DaNode/blob/master/danode/process.d

Danny

Re: Restricting D applications to a single instance

[Preetpal via Digitalmars-d-learn]

On Monday, 22 February 2021 at 08:00:06 UTC, Ali Çehreli wrote:
I achieve it with 'flock' (man 2 flock) on Linux. My case is 
different thoug: It allows me to have single writer and many 
readers of program data, which is kept inside a cache 
directory. All instances start with read permissionss to 
exclude any potential writer. Once a writer grabs the lock for 
writing, no other writer or reader can exist, etc. A nice side 
effect that I realized was since there is only one writer, the 
lock file itself stores progress information so that other 
instances can say something like "Waiting for the writer to 
finish: 75%" by reading from the lock file itself. :)


Works like a charm except flock operations are not atomic; so 
some care is needed. For example, the writer cannot assume 
there were no other writers when it "drops" its access right to 
a reader. It has to check some signature to make sure that 
everything is expected.


Implementing progress reporting in that way is cool. :)

I'll looking into using flock to implement this functionality 
when I port this program to FreeBSD/Linux (I'll probably try 
implementing this functionality in another application first as I 
only infrequently use those operating systems as desktop 
environments).

Re: How add png image to zip file using std.zip?

[Marcone via Digitalmars-d-learn]

On Sunday, 21 February 2021 at 18:10:43 UTC, JN wrote:

On Sunday, 21 February 2021 at 17:17:56 UTC, Marcone wrote:

ZipArchive zip = new ZipArchive();
std.file.write("foo.zip", zip.build());

ArchiveMember f = new ArchiveMember();
f.name = "Wallpaper_001.png";

zip.addMember(f);
std.file.write("foo.zip", zip.build());

File is added with file size 0.
How can I use expandedData for add images files? Or what I use 
for it?


expandedData should hold the contents of the uncompressed file, 
as byte array. So something like:


f.name = "Wallpaper_001.png";
f.expandedData = 
cast(ubyte[])std.file.read("Wallpaper_001.png");


should work. f.name = "Wallpaper_001.png" only says "create a 
file named Wallpaper_001.png inside the zip". It doesn't say 
"put the Wallpaper_001.png file into the zip".


Very good! Work very well. I was trying using rawRead. Now work 
fine.

Re: Class instance alignment

[kinke via Digitalmars-d-learn]

On Monday, 22 February 2021 at 02:23:27 UTC, Steven Schveighoffer 
wrote:

Hm... but does TypeInfo detail alignment?


Apparently not for TypeInfo_Class; .talign() returns the 
alignment of a class *ref*, i.e., pointer size. 
TypeInfo_Struct.talign() does return the struct alignment though 
and could be used to select a larger bin size.

Re: Restricting D applications to a single instance

[Ali Çehreli via Digitalmars-d-learn]

On 2/21/21 7:28 PM, Preetpal wrote:

> I decided to implement what I meant (single application (executable)
> instance restriction) using the Windows API (since my application only
> runs on Windows currently anyways):

I achieve it with 'flock' (man 2 flock) on Linux. My case is different 
thoug: It allows me to have single writer and many readers of program 
data, which is kept inside a cache directory. All instances start with 
read permissionss to exclude any potential writer. Once a writer grabs 
the lock for writing, no other writer or reader can exist, etc. A nice 
side effect that I realized was since there is only one writer, the lock 
file itself stores progress information so that other instances can say 
something like "Waiting for the writer to finish: 75%" by reading from 
the lock file itself. :)


Works like a charm except flock operations are not atomic; so some care 
is needed. For example, the writer cannot assume there were no other 
writers when it "drops" its access right to a reader. It has to check 
some signature to make sure that everything is expected.


Ali