Re: Turning fixed sized array into tuple

2024-05-04 Thread Dmitry Olshansky via Digitalmars-d-learn

On Saturday, 4 May 2024 at 19:11:14 UTC, Nick Treleaven wrote:

On Saturday, 4 May 2024 at 16:58:00 UTC, Dmitry Olshansky wrote:

So I have a function:

```d
size_t awaitAny(T...)(T args) { ... }
```

And I have:
``d
Event*[4] events;
``

How do I pass all 4 of events to awaitAny as tuple of 
arguments?


Use `awaitAny(events.tupleof)`?
https://dlang.org/spec/arrays.html#array-properties


Thanks, totally missed it!

—
Dmitry Olshansky
CEO @ [Glow labs](https://glow-labs.pro)
https://olshansky.me




Turning fixed sized array into tuple

2024-05-04 Thread Dmitry Olshansky via Digitalmars-d-learn

So I have a function:

```d
size_t awaitAny(T...)(T args) { ... }
```

And I have:
``d
Event*[4] events;
``

How do I pass all 4 of events to awaitAny as tuple of arguments?

--
Dmitry Olshansky
CEO @ [Glow labs](https://glow-labs.pro)
https://olshansky.me/about/



Re: Setting up CI for Dub project on Github

2024-04-08 Thread Dmitry Olshansky via Digitalmars-d-learn
On Monday, 8 April 2024 at 13:23:12 UTC, Richard (Rikki) Andrew 
Cattermole wrote:

On 09/04/2024 1:20 AM, Dmitry Olshansky wrote:
I haven’t done any research on the subject, would be nice if 
somebody pointed me to good example of how it’s done.


—
Dmitry Olshansky
CEO @ Glowlabs
https://olshansky.me


In case you haven't already found:

https://github.com/dlang-community/setup-dlang


Thanks, Rikki, that should do the trick.

—
Dmitry Olshansky
CEO @ Glowlabs
https://olshansky.me


Setting up CI for Dub project on Github

2024-04-08 Thread Dmitry Olshansky via Digitalmars-d-learn
I haven’t done any research on the subject, would be nice if 
somebody pointed me to good example of how it’s done.


—
Dmitry Olshansky
CEO @ Glowlabs
https://olshansky.me


Re: Limits of implicit conversion of class arrays

2024-03-23 Thread Dmitry Olshansky via Digitalmars-d-learn

On Saturday, 23 March 2024 at 09:08:45 UTC, Per Nordlöw wrote:

Is there a reason why

```d
class Base {}
class Derived : Base {}

@safe pure nothrow unittest {
Base b;
Derived d;
b = d; // pass

Base[] bs;
Derived[] ds;
bs ~= ds; // pass
bs = ds; // fail [1], should pass
bs = cast(Base[])ds; // fail [2], should pass
}
```

fails as

[1]: cannot implicitly convert expression `ds` of type 
`Derived[]` to `Base[]`

[2]: cast from `Derived[]` to `Base[]` not allowed in safe code

?


The first and second is unsound (infamously allowed in Java). 
Once you cast the slice you can populate it with Derived2 objects 
that are not Derived, hence breaking type safety of the ds slice.


—
Dmitry Olshansky
CEO @ Glow labs
https://olshansky.me



Linker error, doing something wrong?

2023-07-09 Thread Dmitry Olshansky via Digitalmars-d-learn

Trying to compile the following:

https://github.com/DmitryOlshansky/photon/blob/master/tests/curl_download.d

with:
ldc2 curl_download.d -L-lcurl

get:

  "__D6photon12__ModuleInfoZ", referenced from:
  __D13curl_download12__ModuleInfoZ in curl_download.o
  "__D6photon5macos4core2goFDFZvZv", referenced from:
  __D13curl_download4mainFZ13spawnDownloadMFAyaQdZv in 
curl_download.o

  "__D6photon5macos4core9startloopFZv", referenced from:
  __Dmain in curl_download.o
  "__D6photon9runFibersFZv", referenced from:
  __Dmain in curl_download.o
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to 
see invocation)

Error: /usr/bin/cc failed with status: 1

Am I missing something?

--
Dmitry Olshansky
https://olshansky.me



Re: Easiest CI to build on github.com

2023-07-07 Thread Dmitry Olshansky via Digitalmars-d-learn
On Friday, 7 July 2023 at 10:29:14 UTC, Richard (Rikki) Andrew 
Cattermole wrote:

I believe: https://github.com/dlang-community/setup-dlang


Thx!

—
Dmitry Olshansky
CEO @ Glow Labs
https://olshansky.me
https://t.me/glowlabs32


Easiest CI to build on github.com

2023-07-07 Thread Dmitry Olshansky via Digitalmars-d-learn

Simply enough dub test should pass.
How do I go about it?

—
Dmitry Olshansky
CEO @ Glow Labs
https://olshansky.me
https://t.me/glowlabs32


Re: IntelliJ D language plugin

2023-07-02 Thread Dmitry Olshansky via Digitalmars-d-learn
On Friday, 30 June 2023 at 16:26:26 UTC, Richard (Rikki) Andrew 
Cattermole wrote:

I use it and contribute to it ;)


Thanks to Rikki I was able to figure it out.

—
Dmitry Olshansky
https://olshansky.me


IntelliJ D language plugin

2023-06-30 Thread Dmitry Olshansky via Digitalmars-d-learn

Have anyone had any luck with it?

So far I'm trying to install DMD as SDK but it fails with not a 
valid D compiler home.


--
Dmitry Olshansky

https://olshansky.me


Re: Running LDC on a recent MacOS

2023-06-16 Thread Dmitry Olshansky via Digitalmars-d-learn
On Friday, 16 June 2023 at 16:14:19 UTC, Steven Schveighoffer 
wrote:

On 6/16/23 11:56 AM, Dmitry Olshansky wrote:


Any advice from MacOS users?


Yep.

Go into settings, then privacy and security. Make sure "App 
store and identified developers" is checked.


On that page, you will see probably a thing saying "you tried 
to run ldc2 but I blocked it". Say "allow anyway".


Then try it again. This time, it will still block it, but ask 
you if you want to run it.


Yes, thanks that did the trick!



-Steve





Re: Using DUB packages with Meson

2023-04-12 Thread Dmitry Olshansky via Digitalmars-d-learn
On Wednesday, 12 April 2023 at 11:07:56 UTC, Richard (Rikki) 
Andrew Cattermole wrote:

Did you compile the library with dub using ldc2?


Yup, I do not have other compilers installed.

--
Dmitry Olshansky


Re: Using DUB packages with Meson

2023-04-12 Thread Dmitry Olshansky via Digitalmars-d-learn
On Wednesday, 12 April 2023 at 10:24:48 UTC, Richard (Rikki) 
Andrew Cattermole wrote:
I'm going to guess that you need to use the version specifier 
in the package name. Because I'm not seeing anything there to 
handle it specifically.


https://github.com/mesonbuild/meson/blob/master/mesonbuild/dependencies/dub.py

i.e. ``dub build [[@]] []``

So use ``package:sub@1.0.2``.

Also I just noticed meson doesn't support shared libraries from 
dub, so something to keep in mind.


I'm getting closer. I'm stuck with this at the moment:

Found DUB: /home/d.olshanskiy/bin/dub (DUB version 1.31.1, built 
on Mar 12 2023)

ERROR: strand found but it wasn't compiled with ldc
Run-time dependency strand found: NO

src/meson.build:22:0: ERROR: Dependency "strand" not found

--
Dmitry Olshansky


Re: Using DUB packages with Meson

2023-04-12 Thread Dmitry Olshansky via Digitalmars-d-learn
On Wednesday, 12 April 2023 at 10:24:48 UTC, Richard (Rikki) 
Andrew Cattermole wrote:
I'm going to guess that you need to use the version specifier 
in the package name. Because I'm not seeing anything there to 
handle it specifically.


https://github.com/mesonbuild/meson/blob/master/mesonbuild/dependencies/dub.py

i.e. ``dub build [[@]] []``

So use ``package:sub@1.0.2``.

Also I just noticed meson doesn't support shared libraries from 
dub, so something to keep in mind.


Oh, Rikki, you are so helpful. Thanks!

--
Dmitry Olshansky


Using DUB packages with Meson

2023-04-12 Thread Dmitry Olshansky via Digitalmars-d-learn
I'm trying to use my new DUB package from Photon, which is 
polyglot project and is built with Meson.


Have anyone worked with DUB packages in Meson? I've found this 
bit of documentation:

https://mesonbuild.com/Dependencies.html#dub

And I did fetch & build but I do not understand how to introduce 
library dependency on a specific DUB package.


--
Dmitry Olshansky


Re: Help with registering dub package

2023-04-12 Thread Dmitry Olshansky via Digitalmars-d-learn
On Wednesday, 12 April 2023 at 08:25:54 UTC, Richard (Rikki) 
Andrew Cattermole wrote:

Your branches + tag is all messed up.

You have both ~master and ~main. ~main has dub.json (which is 
required), but the tag is based upon ~master which has your 
README.


Between the two branches everything is there, its just that it 
needs to be all in one.


Thanks, Rikki!

—
Dmitry Olshansky




Help with registering dub package

2023-04-12 Thread Dmitry Olshansky via Digitalmars-d-learn

Could someone walk me through the steps of publish my dub package?

I'm stuck with this:
https://code.dlang.org/packages/strand

For some reason code.dlang.org cannot find my semver tag I guess.

--
Dmitry Olshansky


Re: How to build DMD/Phobos on Windows

2022-08-25 Thread Dmitry Olshansky via Digitalmars-d-learn
On Wednesday, 24 August 2022 at 21:11:42 UTC, rikki cattermole 
wrote:

For dmd you use build.d that is in the repository.



Hm, I guess the makefiles should be deleted?


For phobos win64.mak (used for 32bit by default as well):

"# Makefile to build D runtime library phobos{64,32mscoff}.lib 
for Windows MSVC"


So MSVC make.


It’s a shame as I cannot install MSVC on this laptop 
(permissions).




Beyond that idk, but its starting point (oh and druntime is now 
in dmd repo, so ugh... yeah)


Yeah, I see that build become more involved.



How to build DMD/Phobos on Windows

2022-08-24 Thread Dmitry Olshansky via Digitalmars-d-learn
It's been a long time but I've found some spare hours I want to 
devote to finally updating our std.uni to Unicode 14 (soon to 
migrate to 15 I guess).


I downloaded source code of DMD/Phobos as usual and dropped them 
in the src folder of unpacked 7z distribution archive. Now time 
to build. Since DMD doesn't ship make anymore I guessed I needed 
a GNU make so I got that.


Doing make -f win32.mak shows an endless stream of make entering 
the folder and then leaving it.


make[45]: Entering directory 'C:/exp/dmd2/src/dmd'
cd compiler\src
make -f win32.mak
make[46]: Entering directory 'C:/exp/dmd2/src/dmd'
cd compiler\src
make -f win32.mak
make[47]: Entering directory 'C:/exp/dmd2/src/dmd'
cd compiler\src
make -f win32.mak
make[48]: Entering directory 'C:/exp/dmd2/src/dmd'
cd compiler\src
make -f win32.mak
make[49]: Entering directory 'C:/exp/dmd2/src/dmd'
cd compiler\src
make -f win32.mak
make[50]: Entering directory 'C:/exp/dmd2/src/dmd'
cd compiler\src
make -f win32.mak
make[51]: Entering directory 'C:/exp/dmd2/src/dmd'
cd compiler\src
make -f win32.mak

Trying to run any makefiles in compiler/druntime folders do not 
produce anything useful either.


So what is the canonical way to build D on Windows? Any pointers 
would be greately appreciated.




Re: Why The D Style constants are written in camelCase?

2018-05-09 Thread Dmitry Olshansky via Digitalmars-d-learn

On Wednesday, 9 May 2018 at 09:38:14 UTC, BoQsc wrote:
The D Style suggest to camelCase constants, while Java naming 
conventions always promoted uppercase letter.


Is there an explanation why D Style chose to use camelCase 
instead of all UPPERCASE for constants, was there any technical 
problem that would appear while writing in all UPPERCASE?


It is D style for standard library. It is mostly arbitrary but in 
general sensible.

That’s it.





Re: dynamically allocating on the stack

2018-04-21 Thread Dmitry Olshansky via Digitalmars-d-learn

On Saturday, 21 April 2018 at 14:25:58 UTC, Cym13 wrote:

On Saturday, 21 April 2018 at 13:54:14 UTC, H. S. Teoh wrote:
On Sat, Apr 21, 2018 at 01:30:55PM +, Cym13 via 
Digitalmars-d-learn wrote:
On Saturday, 21 April 2018 at 12:08:09 UTC, Dmitry Olshansky 
wrote:

[...]
> Unbounded allocation on stack is kind of anti-pattern and a 
> potential DoS vector.


I'm having trouble seeing how unbounded heap allocations 
aren't equally a potential DoS vector.

[...]

Generally speaking, the heap is much bigger than the stack 
(often many times so) and so is less prone to overflow.  
Though it's true, it still does happen if you just blindly 
allocate memory based on unsanitized external input.



T


Wait, why? Don't they share the same address space and grow in 
opposite directions?


That was true more like 25 years ago. Same address space is 
obviously still true. These days heap is usually not using sbrk 
which is basically what you describe. Also imagine threads and 
tell me which stack grows towards which ;)


Heap is allocated with mmap on Posix’es and VirtualAlloc on 
Windows. Stack is typically fixed limit imposed by ulimit and how 
it grows is beside the point really.







Re: dynamically allocating on the stack

2018-04-21 Thread Dmitry Olshansky via Digitalmars-d-learn

On Saturday, 21 April 2018 at 13:30:55 UTC, Cym13 wrote:
On Saturday, 21 April 2018 at 12:08:09 UTC, Dmitry Olshansky 
wrote:
On Saturday, 21 April 2018 at 07:37:50 UTC, Mike Franklin 
wrote:
Does D have some way to dynamically allocate on the stack?  
I'm looking for something roughly equivalent to the following 
C code.


int doSomething(size_t len)
{
char stackBuffer[len + 1];
doSomethingElse(stackBuffer);
}



Unbounded allocation on stack is kind of anti-pattern and a 
potential DoS vector.


I'm having trouble seeing how unbounded heap allocations aren't 
equally a potential DoS vector.


I could see what you meant, but really stack is typically far 
more limited then heap in size.


Unless you tune the stack size yourself (i.e. you both build an 
app and control the environment) there is no sensible way to know 
the size you can use. It’s also heavily platform dependent.


With heap it’s usually far less limited resource.

Lastly Fibers usually have small stacks.



A separate region allocator is exactly as fast and can easily 
survive across boundaries of function calls.


I guess if OP wants it on the stack it's because it doesn't 
need to survive across boundaries of function calls so this 
buys nothing in this case.


Yet nothing to lose and much safer bet in general.

In general, I’d expect performance to be the goal here. If so 
then zone/region allocation is a well known pattern that is not 
restricted to individual arrays and is widely used in games and 
industry-grade stuff like browsers/VMs.





Also you probably want something like char[X] = void;
 for efficiency if allocating on stack.


Thanks,
Mike





Re: dynamically allocating on the stack

2018-04-21 Thread Dmitry Olshansky via Digitalmars-d-learn

On Saturday, 21 April 2018 at 07:37:50 UTC, Mike Franklin wrote:
Does D have some way to dynamically allocate on the stack?  I'm 
looking for something roughly equivalent to the following C 
code.


int doSomething(size_t len)
{
char stackBuffer[len + 1];
doSomethingElse(stackBuffer);
}



Unbounded allocation on stack is kind of anti-pattern and a 
potential DoS vector.


A separate region allocator is exactly as fast and can easily 
survive across boundaries of function calls.


Also you probably want something like char[X] = void;
 for efficiency if allocating on stack.


Thanks,
Mike





Re: Is socket.send thread safe?

2018-03-26 Thread Dmitry Olshansky via Digitalmars-d-learn

On Monday, 26 March 2018 at 16:14:31 UTC, Jonathan wrote:
Can I send data over an std.socket on multiple threads without 
manual mutexing?


If not, can I send data on a separate thread than receive?

The docs for std.socket say nothing of it (which I guess means 
I should assume it is not thread safe but...).


It’s thin-wrapper over system interface.

It’s thread-safe in that system will not topple over, you will 
likely get some interleaving of data.


HOWEVER: there is such a thing as short-writes, which means that 
since OS buffer may fill up it could write less then you intended 
to and return that amount. It certainly happens with O_NONBLOCK.
In normal blocking mode it might work, but in general it’s a bad 
idea to do writes to a single socket from multiple threads w/o 
some coordination s.t. only can do it at any given time.


Lastly as far as guarantees go, POSIX doesn’t state that 2 writes 
will be interleaved “exactly”, it may for instance splice them 
zebra-style by some chunk size and I could imagine why (on long 
buffers).


My suggestion is to have a queue of messages and a thread to send 
them. You might also want to check and see if there is a better 
protocol then TCP to take advantage of unordered messages. I 
think there is at least RDP (reliable datagram protocol) on Linux 
that might fit the bill.




Thanks!




Re: core.stdc.stdlib._compare_fp_t and qsort

2018-03-18 Thread Dmitry Olshansky via Digitalmars-d-learn

On Sunday, 18 March 2018 at 18:11:02 UTC, Dmitry Olshansky wrote:

On Sunday, 18 March 2018 at 16:45:16 UTC, Joe wrote:

[...]


No it just creates a pair of pointer to recs[0] + length of 
recs, like this:


struct Array
{
  size_t length;
  Record* ptr;
}

In D it’s typed as Record[] and has a number of nice 
properties, like not losing its length;)



[...]


Well since recs is array of pointers this looks like a null 
pointer in your data.


Or rather if 10 is capacity and num_recs is actual length, then:

recs[0..num_recs] is the slice with data that you want to sort.




Re: core.stdc.stdlib._compare_fp_t and qsort

2018-03-18 Thread Dmitry Olshansky via Digitalmars-d-learn

On Sunday, 18 March 2018 at 16:45:16 UTC, Joe wrote:
On Sunday, 18 March 2018 at 13:10:08 UTC, Dmitry Olshansky 
wrote:

Do this to get the usual ptr + length:

sort!((a, b) => to!string((*a).name) < 
to!string((*b).name))(recs[]);


Also to!string would be computed on each compare anew. May 
want to use schwartzSort to avoid that, on 10 elements there 
is no real difference though.


The 10 elements are just because it's a small test program.

What does changing "recs" to "recs[]" as the argument actually 
do?  Does it duplicate the fixed array on the fly? [I guess I 
have to study more!]


No it just creates a pair of pointer to recs[0] + length of recs, 
like this:


struct Array
{
  size_t length;
  Record* ptr;
}

In D it’s typed as Record[] and has a number of nice properties, 
like not losing its length;)


The change does pass the compiler, but at runtime it causes a 
segfault.  The next to last frame in the backtrace shows 
(having changed from to!string to fromStringz and using a 
string template instead of a lambda):


Well since recs is array of pointers this looks like a null 
pointer in your data.


The usual ways to fix that is either print stuff or poke around 
in debugger to see if a Record* is null or .name is null.


#6  0x55565760 in 
std.algorithm.sorting.sort!("fromStringz((*a).name.ptr) < 
fromStringz((*b).name.ptr)", 0, 
testd.Record*[]).sort(testd.Record*[]) (r=...)




Also you don’t need (*a).name, D understands that . can access 
data through pointer (what would be an error in C and require ->).


Then it goes through quickSortImpl, shortSort and sort5, moving 
on to either std.functional.binaryFun or processing of the 
lambda, with a and b equal to 0, ending with a segfault in a ?? 
call from fromStringz or in memcpy called from object._dup (in 
the to!string case).




Re: core.stdc.stdlib._compare_fp_t and qsort

2018-03-18 Thread Dmitry Olshansky via Digitalmars-d-learn

On Sunday, 18 March 2018 at 11:29:47 UTC, Joe wrote:

On Monday, 12 March 2018 at 03:50:42 UTC, Joe wrote:

On Monday, 12 March 2018 at 03:13:08 UTC, Seb wrote:
Out of interest: I wonder what's your usecase for using 
qsort. Or in other words: why you can't use the high-level 
std.algorithm.sorting.sort?


This is only temporary. I will be using 
std.algorithm.sorting.sort. I was converting a C program and 
it annoyed me that I couldn't get the qsort invocation past 
the D compiler.


Now that I'm trying to use std.algorithm.sorting, I'm again 
puzzled by what I need to use for the "less" predicate. My 
first try was:


   sort!((a, b) => to!string((*a).name) < 
to!string((*b).name))(recs);


This results in the error:

Error: template std.algorithm.sorting.sort cannot deduce 
function from argument types !((a, b) => to!string((*a).name) < 
to!string((*b).name))(Record*[10]),


This basically says you have fixed size array, in D they don’t 
decay to slices/pointers for safety reasons (it’s stack memory 
that is easy to leak out of scope with disasterous consequences).


Do this to get the usual ptr + length:

sort!((a, b) => to!string((*a).name) < 
to!string((*b).name))(recs[]);


Also to!string would be computed on each compare anew. May want 
to use schwartzSort to avoid that, on 10 elements there is no 
real difference though.




Re: Networking library

2018-03-15 Thread Dmitry Olshansky via Digitalmars-d-learn

On Thursday, 15 March 2018 at 00:06:49 UTC, Cecil Ward wrote:
Can anyone point me in the direction of a library that provides 
very very lightweight (minimum overhead) asynchronous i/o 
routines for - shopping list


1. sending and receiving IPv4 / IPv6 packets,
2. sending receiving ICMP and
3, handling incoming outgoing TCP connections and
4. handling SCTP connections.



Given the low-level nature of some of your requests, I’d suggest 
seriously look at DPDK.
I don’t know if there is a ready-made D wrapper, but it should be 
stright-forward to bind to it. That covers all of your direct 
packet manipulation at extreme speeds.


Re: Garbage collected pointers?

2018-03-01 Thread Dmitry Olshansky via Digitalmars-d-learn

On Thursday, 1 March 2018 at 10:10:27 UTC, John Burton wrote:

In the language spec here :-
https://dlang.org/spec/garbage.html#pointers_and_gc

It refers to a distinction between pointers to garbage 
collected memory and pointers that are not. In particular it 
says that with a non garbage collected pointer you can do 
anything  that is legal in C but with a garbage collected 
pointer there are a lot of undefined behaviors if you don't 
follow some restrictions.


Basically you can do whatever you want with pointers, you may 
twart ability of GC to free an object b/c you constructed some 
value that points to GC heap. That’s pretty much it.


Second side is loosing pointers to GC owned object, this happens 
when you put pointers into eg malloced chunk of memory.


Calling GC.addRange/removeRange you can let GC know that you 
stored pointers to GC heap in some place other then stack or GC 
heap.






Re: forcing tabs in regex

2018-02-27 Thread Dmitry Olshansky via Digitalmars-d-learn
On Wednesday, 28 February 2018 at 05:09:03 UTC, psychoticRabbit 
wrote:

On Wednesday, 28 February 2018 at 01:06:30 UTC, dark777 wrote:

Regex validates years bisexto and not bisextos in format:
const std::regex 
pattern(R"(^(?:(?:(0?[1-9]|1\d|2[0-8])([-/.])(0?[1-9]|1[0-2]|[Jj](?:an|u[nl])|[Mm]a[ry]|[Aa](?:pr|ug)|[Ss]ep|[Oo]ct|[Nn]ov|[Dd]ec|[Ff]eb)|(29|30)([-/.])(0?[13-9]|1[0-2]|[Jj](?:an|u[nl])|[Mm]a[ry]|[Aa](?:pr|ug)|[Ss]ep|[Oo]ct|[Nn]ov|[Dd]ec)|(31)([-/.])(0?[13578]|1[02]|[Jj]an|[Mm]a[ry]|[Jj]ul|[Aa]ug|[Oo]ct|[Dd]ec))(?:\2|\5|\8)(0{2,3}[1-9]|0{1,2}[1-9]\d|0?[1-9]\d{2}|[1-9]\d{3})|(29)([-/.])(0?2|[Ff]eb)\12(\d{1,2}(?:0[48]|[2468][048]|[13579][26])|(?:0?[48]|[13579][26]|[2468][048])00))$)");


this regex above validates the formats through backreferences.



what is this evil dark magic?


Something that is horribly slow and might be incorrect, there are 
very few reasons to write large regexes like that and they 
usually boil down to “it only accepts regex” otherwise parser 
combinators are much better fit.





Re: multithread/concurrency/parallel methods and performance

2018-02-19 Thread Dmitry Olshansky via Digitalmars-d-learn

On Monday, 19 February 2018 at 14:57:22 UTC, SrMordred wrote:
On Monday, 19 February 2018 at 05:54:53 UTC, Dmitry Olshansky 
wrote:
The operation is trivial and dataset is rather small. In such 
cases SIMD with eg array ops is the way to go:

result[] = values[] * values2[];


Yes, absolutely right :)

I make a simple example to understand why the threads are not 
scaling in the way i thought they would.


Yeah, the world is ugly place where trivial math sometimes 
doesn’t work.


I suggest to:
- run with different number of threads from 1 to n
- vary sizes from 100k to 10m

For your numbers - 400ms / 64 is ~ 6ms, if we divide by # cores 
it’s 6/7 ~ 0.86ms which is a deal smaller then a CPU timeslice.


In essence a single core runs fast b/c it doesn’t wait for all 
others to complete via join easily burning its quota in one go. 
In MT I bet some of overhead comes from not all threads finishing 
(and starting) at once, so the join block in the kernel.


You could run your MT code with strace to see if it hits the 
futex call or some such, if it does that’s where you are getting 
delays. (that’s assuming you are on Linux)


std.parallel version is a bit faster b/c I think it caches 
created threadpool so you don’t start threads anew on each run.


I imagine that, if one core work is done in 200ms a 4 core work 
will be done in 50ms, plus some overhead, since they are 
working on separate block of memory, without need of sync, and 
without false sharing, etc (at least I think i don´t have this 
problem here).


If you had a long queue of small tasks like that and you don’t 
wait to join all threads untill absolutely required you get near 
perfect scalability. (Unless hitting other bottlenecks like RAM).






Re: multithread/concurrency/parallel methods and performance

2018-02-18 Thread Dmitry Olshansky via Digitalmars-d-learn

On Sunday, 18 February 2018 at 17:54:58 UTC, SrMordred wrote:

I´m experimenting with threads and related recently.
(i´m just started so may be some terrrible mistakes here)

With this base work:

foreach(i ; 0 .. SIZE)
{
results[i] = values1[i] * values2[i];
}

and then with this 3 others methods: parallel, spawn and 
Threads.


this was my results:

_base : 456 ms and 479 us
_parallel : 331 ms, 324 us, and 4 hnsecs
_concurrency : 367 ms, 348 us, and 2 hnsecs
_thread : 369 ms, 565 us, and 3 hnsecs

(code here : https://run.dlang.io/is/2pdmmk )

All methods have minor speedup gains.  I was expecting a lot 
more.

Since I have 7 cores I expected like below 100ms.


The operation is trivial and dataset is rather small. In such 
cases SIMD with eg array ops is the way to go:

result[] = values[] * values2[];


Parallelism is gets more interesting with more expensive 
operations.  You may also try bigger sizes or both.


I´m not seeing false sharing in this case. or i'm wrong?

If someone can expand on this, i'll be grateful.

Thanks!





Re: Get largest heap object at runtime? ...tracking the leak

2018-01-21 Thread Dmitry Olshansky via Digitalmars-d-learn

On Sunday, 21 January 2018 at 17:28:13 UTC, Andres Clari wrote:
Hi, is there any way to get from the GC all allocated objects, 
so I can see their size and find where I'm leaking memory? Or 
perhaps a good tool to help with this issue...


I tried building my program with "profile-gc" but I got an 
invalid MemoryOperationError with no stack trace... so no luck 
using that. And the leak is only obvious when running in 
production, where after 6 hours I've seen 1.5Gb of memory usage.



Are you by chance using vibe.d?
In versions 0.8.x I observe a huge memory leak. Might be your 
problem.




Right now, I'm able to extend the life of my program by 
injecting calls to:

GC.collect
GCminimize

Within an event that happens every 5 seconds, which I estimate 
will give me about a week before the program's memory is again 
at +1.5Gb.





D on Power8 (PPC64)

2017-08-07 Thread Dmitry Olshansky via Digitalmars-d-learn

What is the status of the platform?
I might be doing some number crunching on one of the power8 
beasts, would be nice to have D working there.


---
Dmitry Olshansky


Re: Is it ok to inherit multiple times same templated interface?

2017-01-15 Thread Dmitry Olshansky via Digitalmars-d-learn
On Sunday, 15 January 2017 at 20:33:30 UTC, Alexandru Ermicioi 
wrote:

Good day,

Given following code example, where a templated interface Wr, 
and an implementation Im is present:



From the standpoint of the compiler they are 3 distinct 
interfaces, so all is good.



interface Wr(T) {
T get();
}

[...]




Re: Build your own Trie entry table

2016-12-27 Thread Dmitry Olshansky via Digitalmars-d-learn

On Monday, 26 December 2016 at 10:12:20 UTC, Remi Thebault wrote:

Hello

I want to map a dchar to its Bidi_Class.

I've built an utility that reads UnicodeData.txt into an AA and 
builds a trie with std.uni.codepointTrie from it.
I use Trie.store() to export the trie entry table into a D 
module. (I believe in a similar manner than phobos' 
unicode_tables.d)


Now I want to use this table to efficiently create a Trie in my 
code, the same way std.uni does, but found out that Trie 
constructor is private.


I've copy-pasted the asTrie function (and also TrieEntry 
struct) in my code and it works well (which I don't understand 
because my code still ends up calling the private ctor).




This must be an oversight on my part. Indeed it should be 
possible to store/load your own trie.


Can you give indication on the workflow one should follow for 
this use case?


Please file a bug report for Phobos basically stating your use 
case and the visibility problem.


Thanks
Rémi



Dmitry Olshansky




Re: Degenerate Regex Case

2015-04-25 Thread Dmitry Olshansky via Digitalmars-d-learn

On Friday, 24 April 2015 at 18:28:16 UTC, Guillaume wrote:
Hello, I'm trying to make a regex comparison with D, based off 
of this article: https://swtch.com/~rsc/regexp/regexp1.html


I've written my code like so:

import std.stdio, std.regex;

void main(string argv[]) {

string m = argv[1];
	auto p = 
ctRegex!("a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?aaa");

if (match(m, p)) {
writeln("match");
} else {
writeln("no match");
}

}




And the compiler goes into swap. Doing it at runtime is no 
better. I was under the impression that this particular regex 
was used for showcasing the Thompson NFA which D claims to be 
using.




A quick investigation shows that it gets stuck at the end of 
pattern compilation stage.


The problem is that as a last pass D's regex goes to optimize the 
pattern to construct simple bit-scanning engine as approximation 
for prefix of original pattern. And that process is a lot like 
Thompson NFA ... _BUT_ the trick of merging equivalent threads 
wasn't applied there.


So in short: file a bug, optimizer absolutely should do 
de-duplication of threads.



The golang code version of this runs fine, which makes me think 
that maybe D isn't using the correct regex engine for this 
particular regex. Or perhaps I'm using this wrong?


It uses 2 kinds of engines, run-time one is Thompson NFA. 
Compile-time is (for now) still backtracking.


---
Dmitry Olshansky