Re: Accessing x86 Performance Counters
On Wednesday, 13 May 2015 at 03:38:33 UTC, Maxime Chevalier-Boisvert wrote: I was wondering if anyone has written D code to access the x86 performance counters, to get information such as the number of cache misses and cycle count. I considered doing that at one point. So I looked for resources on the topic and it turns out that you need a kernel mode executable to be able to read the performance counters. Luckly there is a open source driver for windows which gives you some API to access the counters from user mode executables. Unfortunately you need to either switch your windows into a "unsafe mode" to allow for loading unsigned drivers or you need to somehow get a certificate to sign the driver (these are quite expensive). At that point I stopped looking into this, because both of these options weren't viable for my use case. Once I find some time I could dig up the resources I found if you are interested.
Accessing x86 Performance Counters
I was wondering if anyone has written D code to access the x86 performance counters, to get information such as the number of cache misses and cycle count.
Re: problem with parallel foreach
On Tuesday, 12 May 2015 at 20:50:45 UTC, Gerald Jansen wrote: Your advice is appreciated but quite disheartening. I was hoping for something (nearly) as easy to use as Python's parallel.Pool() map(), given that this is essentially an "embarassingly parallel" problem. Avoidance of GC allocation and self-written buffered IO functions seems a bit much to ask of a newcomer to a language. You're right, these are issues of D's standard library that are not easy for a newcomer to tackle. In case of Python's parallel.Pool() separate processes do the work without any synchronization issues. In case of D's std.parallelism it's just threads inside one process and they do fight for some locks, thus this result.
Re: SysTime object from unixtime
see
import std.stdio;
import std.datetime;
import std.process;
void main() {
auto ct = Clock.currTime;
write("\n\n\tData: ",ct.day," / ",ct.month," / ",ct.year,".\n");
auto tv = Clock.currTime;
write("\tHour: ",
tv.hour,":",
tv.minute,":",
tv.second,".",
tv.fracSec,"\n ");
writeln("\n");
wait(spawnShell("pause"));
}
Re: SysTime object from unixtime
The "interfacing to C" header of this page talks about it:
http://dlang.org/intro-to-datetime.html
import std.datetime;
import core.stdc.time;
void main() {
time_t unixTime = core.stdc.time.time(null);
auto stdTime = unixTimeToStdTime(unixTime);
auto st = SysTime(stdTime);
}
I wish std.datetime made this a bit easier but that's how to do
it.
Re: SysTime object from unixtime
On Wednesday, 13 May 2015 at 01:40:04 UTC, Adam D. Ruppe wrote: The "interfacing to C" Sorry, "interfacing with C", search for those words on that page and it will bring you to the header.
SysTime object from unixtime
As strange as it sounds, after 20 minutes reading the docs about std.datetime I could not figure out how to get a SysTime object starting from a long unixtime. I'm sure I'm missing something obvious, any tips? Thanks, dat
Re: problem with parallel foreach
On Tuesday, 12 May 2015 at 20:58:16 UTC, Vladimir Panteleev wrote:
On Tuesday, 12 May 2015 at 18:14:56 UTC, Gerald Jansen wrote:
On Tuesday, 12 May 2015 at 16:35:23 UTC, Rikki Cattermole
wrote:
On 13/05/2015 4:20 a.m., Gerald Jansen wrote:
At the risk of great embarassment ... here's my program:
http://dekoppel.eu/tmp/pedupg.d
Would it be possible to give us some example data?
I might give it a go to try rewriting it tomorrow.
http://dekoppel.eu/tmp/pedupgLarge.tar.gz (89 Mb)
Contains two largish datasets in a directory structure
expected by the program.
Profiling shows that your program spends most of the time
reading the data.
I see a considerable speed boost with the following one-line
patch (plus imports):
- foreach (line; File(pednum, "r").byLine()) {
+ foreach (line; (cast(string)read(pednum)).split('\n')) {
Nice, thanks. Making that replacement in three points in the
program resulted in roughly a 30% speedup at the cost of about
30% more memory in this specific case. Unfortunately it didn't
help with the performance deteroration problem with parallel
foreach.
Re: Binding to C
On Tuesday, 12 May 2015 at 03:20:34 UTC, TJB wrote: I'm sure this question has been asked a thousand times. I've even asked similar questions in the past (I've been away for quite a while). But all of the tutorials that I have found assume quite a lot and leave a lot to be inferred. Is there a simple step by step tutorial demonstrating how to call C code from D? With the separate file contents and names and compilation instructions? Thanks, TJB Hello if you work with Windows and has the vs2012 a look at this post https://github.com/cassio2014/DIC/tree/master/dic.v2/exemplos/dll%20in%20vs2012%20Cpp%20%20and%20read%20in%20D
Re: problem with parallel foreach
On Tuesday, 12 May 2015 at 18:14:56 UTC, Gerald Jansen wrote:
On Tuesday, 12 May 2015 at 16:35:23 UTC, Rikki Cattermole wrote:
On 13/05/2015 4:20 a.m., Gerald Jansen wrote:
At the risk of great embarassment ... here's my program:
http://dekoppel.eu/tmp/pedupg.d
Would it be possible to give us some example data?
I might give it a go to try rewriting it tomorrow.
http://dekoppel.eu/tmp/pedupgLarge.tar.gz (89 Mb)
Contains two largish datasets in a directory structure expected
by the program.
Profiling shows that your program spends most of the time reading
the data.
I see a considerable speed boost with the following one-line
patch (plus imports):
- foreach (line; File(pednum, "r").byLine()) {
+ foreach (line; (cast(string)read(pednum)).split('\n')) {
Re: problem with parallel foreach
On Tuesday, 12 May 2015 at 17:45:54 UTC, thedeemon wrote: On Tuesday, 12 May 2015 at 17:02:19 UTC, Gerald Jansen wrote: About 3.5 million lines read by main(), 0.5 to 2 million lines read and 3.5 million lines written by runTraits (aka runJob). Each GC allocation in D is a locking operation (and disabling GC doesn't help here at all), probably each writeln too, so when multiple threads try to write millions of lines such issue is easy to meet. I would look for a way to write those lines without allocations and locking, and also reduce total number of system calls by buffering data, doing less f.writef's. Your advice is appreciated but quite disheartening. I was hoping for something (nearly) as easy to use as Python's parallel.Pool() map(), given that this is essentially an "embarassingly parallel" problem. Avoidance of GC allocation and self-written buffered IO functions seems a bit much to ask of a newcomer to a language.
Re: problem with parallel foreach
On Tuesday, 12 May 2015 at 19:14:23 UTC, Laeeth Isharc wrote: But if you disable the logging does that change things? There is only a tiny bit of logging happening. And are you using optimization on gdc ? gdc -Ofast -march=native -frelease Also try byLineFast eg http://forum.dlang.org/thread/umkcjntsxchskljyg...@forum.dlang.org#post-20130516144627.50da:40unknown Thx, I'll have a look. Performance is good for a single dataset so I thought regular byLine would be okay. I don't know if std.csv CSVReader would be faster than parsing yourself, but worth trying. No, my initial experience with CSVReader was that it was not very fast: http://forum.dlang.org/post/wklmolsqcmnagluid...@forum.dlang.org . Some tricks here, also: http://tech.adroll.com/blog/data/2014/11/17/d-is-for-data-science.html Thanks again. I am having doubts about "d-is-for-data-science". The learning curve is very steep compared to my experience with R/Python/(Julia). But I'm trying...
Re: problem with parallel foreach
On Tuesday, 12 May 2015 at 19:10:13 UTC, Laeeth Isharc wrote: On Tuesday, 12 May 2015 at 18:14:56 UTC, Gerald Jansen wrote: On Tuesday, 12 May 2015 at 16:35:23 UTC, Rikki Cattermole wrote: On 13/05/2015 4:20 a.m., Gerald Jansen wrote: At the risk of great embarassment ... here's my program: http://dekoppel.eu/tmp/pedupg.d Would it be possible to give us some example data? I might give it a go to try rewriting it tomorrow. http://dekoppel.eu/tmp/pedupgLarge.tar.gz (89 Mb) Contains two largish datasets in a directory structure expected by the program. I haven't had time to read code closely. But if you disable the logging does that change things? If so, how about having the logging done asynchronously in another thread? And are you using optimization on gdc ? Also try byLineFast eg http://forum.dlang.org/thread/umkcjntsxchskljyg...@forum.dlang.org#post-20130516144627.50da:40unknown I don't know if std.csv CSVReader would be faster than parsing yourself, but worth trying. Some tricks here, also: http://tech.adroll.com/blog/data/2014/11/17/d-is-for-data-science.html
Re: problem with parallel foreach
On Tuesday, 12 May 2015 at 18:14:56 UTC, Gerald Jansen wrote: On Tuesday, 12 May 2015 at 16:35:23 UTC, Rikki Cattermole wrote: On 13/05/2015 4:20 a.m., Gerald Jansen wrote: At the risk of great embarassment ... here's my program: http://dekoppel.eu/tmp/pedupg.d Would it be possible to give us some example data? I might give it a go to try rewriting it tomorrow. http://dekoppel.eu/tmp/pedupgLarge.tar.gz (89 Mb) Contains two largish datasets in a directory structure expected by the program. I haven't had time to read code closely. But if you disable the logging does that change things? If so, how about having the logging done asynchronously in another thread? And are you using optimization on gdc ?
Re: problem with parallel foreach
On Tuesday, 12 May 2015 at 16:35:23 UTC, Rikki Cattermole wrote: On 13/05/2015 4:20 a.m., Gerald Jansen wrote: At the risk of great embarassment ... here's my program: http://dekoppel.eu/tmp/pedupg.d Would it be possible to give us some example data? I might give it a go to try rewriting it tomorrow. http://dekoppel.eu/tmp/pedupgLarge.tar.gz (89 Mb) Contains two largish datasets in a directory structure expected by the program.
Re: problem with parallel foreach
On Tuesday, 12 May 2015 at 17:02:19 UTC, Gerald Jansen wrote: About 3.5 million lines read by main(), 0.5 to 2 million lines read and 3.5 million lines written by runTraits (aka runJob). Each GC allocation in D is a locking operation (and disabling GC doesn't help here at all), probably each writeln too, so when multiple threads try to write millions of lines such issue is easy to meet. I would look for a way to write those lines without allocations and locking, and also reduce total number of system calls by buffering data, doing less f.writef's.
Re: problem with parallel foreach
On Tuesday, 12 May 2015 at 16:46:42 UTC, thedeemon wrote: On Tuesday, 12 May 2015 at 14:59:38 UTC, Gerald Jansen wrote: The output of /usr/bin/time is as follows: Lang JobsUser System Elapsed %CPU Py 2 79.242.16 0:48.90 166 D 2 19.41 10.14 0:17.96 164 Py 30 1255.17 58.38 2:39.54 823 * Pool(12) D 30 421.61 4565.97 6:33.73 1241 The fact that most of the time is spent in System department is quite important. I suspect there are too many system calls from line-wise reading and writing the files. How many lines are read and written there? About 3.5 million lines read by main(), 0.5 to 2 million lines read and 3.5 million lines written by runTraits (aka runJob). I have smaller datasets that I test on my laptop with a single quad-core I7 which sometimes show little increase in System time and other times have a marked increase, but not nearly as exagerated as in the large datasets on the server. Gerald
Re: problem with parallel foreach
On Tuesday, 12 May 2015 at 14:59:38 UTC, Gerald Jansen wrote: The output of /usr/bin/time is as follows: Lang JobsUser System Elapsed %CPU Py 2 79.242.16 0:48.90 166 D 2 19.41 10.14 0:17.96 164 Py 30 1255.17 58.38 2:39.54 823 * Pool(12) D 30 421.61 4565.97 6:33.73 1241 The fact that most of the time is spent in System department is quite important. I suspect there are too many system calls from line-wise reading and writing the files. How many lines are read and written there?
Re: problem with parallel foreach
On 13/05/2015 4:20 a.m., Gerald Jansen wrote: At the risk of great embarassment ... here's my program: http://dekoppel.eu/tmp/pedupg.d As per Rick's first suggestion (thanks) I added import core.memory : GC; main() GC.disable; GC.reserve(1024 * 1024 * 1024); ... to no avail. thanks for all the help so far. Gerald ps. I am using GDC 4.9.2 and don't have DMD on that server Well atleast we now know that it isn't caused by memory allocation/deallocation! Would it be possible to give us some example data? I might give it a go to try rewriting it tomorrow.
Re: problem with parallel foreach
At the risk of great embarassment ... here's my program: http://dekoppel.eu/tmp/pedupg.d As per Rick's first suggestion (thanks) I added import core.memory : GC; main() GC.disable; GC.reserve(1024 * 1024 * 1024); ... to no avail. thanks for all the help so far. Gerald ps. I am using GDC 4.9.2 and don't have DMD on that server
Re: problem with parallel foreach
On 05/12/2015 08:35 AM, Gerald Jansen wrote: > I could put it somewhere if that would help. Please do so. We all want to learn to avoid such issues. Thank you, Ali
Re: problem with parallel foreach
On 13/05/2015 2:59 a.m., Gerald Jansen wrote:
I am a data analyst trying to learn enough D to decide whether to use D
for a new project rather than Python + Fortran. I have recoded a
non-trivial Python program to do some simple parallel data processing
(using the map function in Python's multiprocessing module and parallel
foreach in D). I was very happy that my D version ran considerably
faster that Python version when running a single job but was soon
dismayed to find that the performance of my D version deteriorates
rapidly beyond a handful of jobs whereas the time for the Python version
increases linearly with the number of jobs per cpu core.
The server has 4 quad-core Xeons and abundant memory compared to my
needs for this task even though there are several million records in
each dataset. The basic structure of the D program is:
import std.parallelism; // and other modules
function main()
{
// ...
// read common data and store in arrays
// ...
foreach (job; parallel(jobs, 1)) {
runJob(job, arr1, arr2.dup);
}
}
function runJob(string job, in int[] arr1, int[] arr2)
{
// read file of job specific data file and modify arr2 copy
// write job specific output data file
}
The output of /usr/bin/time is as follows:
Lang JobsUser System Elapsed %CPU
Py 1 45.171.44 0:46.65 99
D 18.441.17 0:09.24 104
Py 2 79.242.16 0:48.90 166
D 2 19.41 10.14 0:17.96 164
Py 30 1255.17 58.38 2:39.54 823 * Pool(12)
D 30 421.61 4565.97 6:33.73 1241
(Note that the Python program was somewhat optimized with numpy
vectorization and a bit of numba jit compilation.)
The system time varies widely between repititions for D with multiple
jobs (eg. from 3.8 to 21.5 seconds for 2 jobs).
Clearly simple my approach with parallel foreach has some problem(s).
Any suggestions?
Gerald Jansen
A couple of things comes to mind at the start of the main function.
---
import core.runtime : GC;
GC.disable;
GC.reserve(1024 * 1024 * 1024);
--
That will reserve 1gb of ram for the GC to work with. It will also stop
the GC from trying to collect.
I would HIGHLY recommend that each worker thread have a preallocated
amount of memory for it. Where that memory is then used to have the data
put into.
Basically you are just being sloppy memory allocation wise.
Try your best to move your processing into @nogc annotated functions.
Then make it as much as possible with no allocations.
Remember you can pass around slices of arrays as long as they are not
mutated without memory allocation freely!
Re: problem with parallel foreach
Thanks Ali. I have tried putting GC.disable() in both main and runJob, but the timing behaviour did not change. The python version works in a similar fashion and also has automatic GC. I tend to think that is not the (biggest) problem. The program is long and newbie-ugly ... but I could put it somewhere if that would help. Gerald On Tuesday, 12 May 2015 at 15:24:45 UTC, Ali Çehreli wrote: On 05/12/2015 07:59 AM, Gerald Jansen wrote: > the performance of my D version deteriorates > rapidly beyond a handful of jobs whereas the time for the Python version > increases linearly with the number of jobs per cpu core. It may be related to GC collections. If it hasn't been changed recently, every allocation from GC triggers a collection cycle. D's current GC being a stop-the-world kind, you lose all benefit of parallel processing when that happens. Without seeing runJob, even arr2.dup may be having such an effect on the performance. Ali
Re: problem with parallel foreach
On 05/12/2015 07:59 AM, Gerald Jansen wrote: > the performance of my D version deteriorates > rapidly beyond a handful of jobs whereas the time for the Python version > increases linearly with the number of jobs per cpu core. It may be related to GC collections. If it hasn't been changed recently, every allocation from GC triggers a collection cycle. D's current GC being a stop-the-world kind, you lose all benefit of parallel processing when that happens. Without seeing runJob, even arr2.dup may be having such an effect on the performance. Ali
Re: problem with parallel foreach
On Tuesday, 12 May 2015 at 15:11:01 UTC, John Colvin wrote:
On Tuesday, 12 May 2015 at 14:59:38 UTC, Gerald Jansen wrote:
I am a data analyst trying to learn enough D to decide whether
to use D for a new project rather than Python + Fortran. I
have recoded a non-trivial Python program to do some simple
parallel data processing (using the map function in Python's
multiprocessing module and parallel foreach in D). I was very
happy that my D version ran considerably faster that Python
version when running a single job but was soon dismayed to
find that the performance of my D version deteriorates rapidly
beyond a handful of jobs whereas the time for the Python
version increases linearly with the number of jobs per cpu
core.
The server has 4 quad-core Xeons and abundant memory compared
to my needs for this task even though there are several
million records in each dataset. The basic structure of the D
program is:
import std.parallelism; // and other modules
function main()
{
// ...
// read common data and store in arrays
// ...
foreach (job; parallel(jobs, 1)) {
runJob(job, arr1, arr2.dup);
}
}
function runJob(string job, in int[] arr1, int[] arr2)
{
// read file of job specific data file and modify arr2 copy
// write job specific output data file
}
The output of /usr/bin/time is as follows:
Lang JobsUser System Elapsed %CPU
Py 1 45.171.44 0:46.65 99
D 18.441.17 0:09.24 104
Py 2 79.242.16 0:48.90 166
D 2 19.41 10.14 0:17.96 164
Py 30 1255.17 58.38 2:39.54 823 * Pool(12)
D 30 421.61 4565.97 6:33.73 1241
(Note that the Python program was somewhat optimized with
numpy vectorization and a bit of numba jit compilation.)
The system time varies widely between repititions for D with
multiple jobs (eg. from 3.8 to 21.5 seconds for 2 jobs).
Clearly simple my approach with parallel foreach has some
problem(s). Any suggestions?
Gerald Jansen
Have you tried adjusting the workUnitSize argument to parallel?
It should probably be 1 for such large individual tasks.
ignore me, i missed that you already had done that.
Re: problem with parallel foreach
On Tuesday, 12 May 2015 at 14:59:38 UTC, Gerald Jansen wrote:
I am a data analyst trying to learn enough D to decide whether
to use D for a new project rather than Python + Fortran. I
have recoded a non-trivial Python program to do some simple
parallel data processing (using the map function in Python's
multiprocessing module and parallel foreach in D). I was very
happy that my D version ran considerably faster that Python
version when running a single job but was soon dismayed to find
that the performance of my D version deteriorates rapidly
beyond a handful of jobs whereas the time for the Python
version increases linearly with the number of jobs per cpu core.
The server has 4 quad-core Xeons and abundant memory compared
to my needs for this task even though there are several million
records in each dataset. The basic structure of the D program
is:
import std.parallelism; // and other modules
function main()
{
// ...
// read common data and store in arrays
// ...
foreach (job; parallel(jobs, 1)) {
runJob(job, arr1, arr2.dup);
}
}
function runJob(string job, in int[] arr1, int[] arr2)
{
// read file of job specific data file and modify arr2 copy
// write job specific output data file
}
The output of /usr/bin/time is as follows:
Lang JobsUser System Elapsed %CPU
Py 1 45.171.44 0:46.65 99
D 18.441.17 0:09.24 104
Py 2 79.242.16 0:48.90 166
D 2 19.41 10.14 0:17.96 164
Py 30 1255.17 58.38 2:39.54 823 * Pool(12)
D 30 421.61 4565.97 6:33.73 1241
(Note that the Python program was somewhat optimized with numpy
vectorization and a bit of numba jit compilation.)
The system time varies widely between repititions for D with
multiple jobs (eg. from 3.8 to 21.5 seconds for 2 jobs).
Clearly simple my approach with parallel foreach has some
problem(s). Any suggestions?
Gerald Jansen
Have you tried adjusting the workUnitSize argument to parallel?
It should probably be 1 for such large individual tasks.
problem with parallel foreach
I am a data analyst trying to learn enough D to decide whether to
use D for a new project rather than Python + Fortran. I have
recoded a non-trivial Python program to do some simple parallel
data processing (using the map function in Python's
multiprocessing module and parallel foreach in D). I was very
happy that my D version ran considerably faster that Python
version when running a single job but was soon dismayed to find
that the performance of my D version deteriorates rapidly beyond
a handful of jobs whereas the time for the Python version
increases linearly with the number of jobs per cpu core.
The server has 4 quad-core Xeons and abundant memory compared to
my needs for this task even though there are several million
records in each dataset. The basic structure of the D program is:
import std.parallelism; // and other modules
function main()
{
// ...
// read common data and store in arrays
// ...
foreach (job; parallel(jobs, 1)) {
runJob(job, arr1, arr2.dup);
}
}
function runJob(string job, in int[] arr1, int[] arr2)
{
// read file of job specific data file and modify arr2 copy
// write job specific output data file
}
The output of /usr/bin/time is as follows:
Lang JobsUser System Elapsed %CPU
Py 1 45.171.44 0:46.65 99
D 18.441.17 0:09.24 104
Py 2 79.242.16 0:48.90 166
D 2 19.41 10.14 0:17.96 164
Py 30 1255.17 58.38 2:39.54 823 * Pool(12)
D 30 421.61 4565.97 6:33.73 1241
(Note that the Python program was somewhat optimized with numpy
vectorization and a bit of numba jit compilation.)
The system time varies widely between repititions for D with
multiple jobs (eg. from 3.8 to 21.5 seconds for 2 jobs).
Clearly simple my approach with parallel foreach has some
problem(s). Any suggestions?
Gerald Jansen
Re: Run-time Indexing of a Compile-Time Tuple
On Monday, 11 May 2015 at 22:46:00 UTC, Per Nordlöw wrote:
The pattern
final switch (_index)
{
import std.range: empty, front;
foreach (i, R; Rs)
{
case i:
assert(!source[i].empty);
return source[i].front;
}
}
occurring in roundRobin() and now also in my merge at
https://github.com/nordlow/justd/blob/master/range_ex.d#L604
is nor pretty nor concise.
Is there a better way of indexing a compile time tuple with a
run-time index?
It may have to work together with CommonType somehow as is
shown in my implementation of merge().
A little hacky, but how about casting it to a static array?
http://dpaste.dzfl.pl/d0059e6e6c09
Re: Clean way to tell whether a destructor is called by the GC
On Tuesday, 12 May 2015 at 12:31:35 UTC, Adam D. Ruppe wrote: Let me suggest a completely different option: make a destructor that works while the GC is running by managing the resources manually in both construction and destruction. I see, but not all ressources can be disposed by any thread. I'd prefer to make it an error. Also, relying on GC destructors can make a program work by mistake. isn't *all* reference types that are problematic to access, it is just GC managed references. So anything from a C function is cool. Otherwise, accessing the runtime variable seems ok to me like the link said, hacky sure, but it'd work nothing else comes to mind, other than using a separate dispose() method when you delete it manually and leave the dtor for only the GC. I already have such a dispose() function. The problem is that to support Unique! and scoped! and friends, the destructor must call dispose(). Thus my need for a way to separate the GC-induced destructors within dispose() or ~this (same problem).
Re: Clean way to tell whether a destructor is called by the GC
Let me suggest a completely different option: make a destructor that works while the GC is running by managing the resources manually in both construction and destruction. Remember, it isn't *all* reference types that are problematic to access, it is just GC managed references. So anything from a C function is cool. Otherwise, accessing the runtime variable seems ok to me like the link said, hacky sure, but it'd work nothing else comes to mind, other than using a separate dispose() method when you delete it manually and leave the dtor for only the GC.
Clean way to tell whether a destructor is called by the GC
I need a robust "isCalledByGC" function to leak intentionally
when a destructor is called as a result of the GC.
It is a way to enforce deterministic destruction without ever
relying on accidental correctness.
class A
{
~this()
{
if(iscalledByGC())
{
[leak and write an error to stderr]
}
else
{
[normal release of resources]
}
}
}
Prior solution:
http://forum.dlang.org/post/li2oj4$1avo$1...@digitalmars.com but
that involves modifying the runtime.
Re: Run-time Indexing of a Compile-Time Tuple
On Monday, 11 May 2015 at 23:12:17 UTC, Alex Parrill wrote: Would this work? auto x = std.typecons.tuple(Rs)[_index] Nope. _index must be know at compile-time.
Re: Run-time Indexing of a Compile-Time Tuple
On Monday, 11 May 2015 at 22:46:00 UTC, Per Nordlöw wrote:
The pattern
final switch (_index)
{
import std.range: empty, front;
foreach (i, R; Rs)
{
case i:
assert(!source[i].empty);
return source[i].front;
}
}
occurring in roundRobin() and now also in my merge at
https://github.com/nordlow/justd/blob/master/range_ex.d#L604
is nor pretty nor concise.
Is there a better way of indexing a compile time tuple with a
run-time index?
It may have to work together with CommonType somehow as is
shown in my implementation of merge().
I wrote that code in roundRobin to replace a nightmare string
mixin. I can't see any way of getting around it, as there is no
meaningful CommonType for a tuple of arbitrary ranges. The body
of each case statement needs to know the index at compile-time.
If the tuple genuinely did have a CommonType, then it would be
easy to make a little free function (or member of
std.typecons.tuple) to get this sort of result:
import std.typecons, std.traits;
CommonType!(T.Types) rtIdx(T)(T t, uint i)
if(is(T : Tuple!A, A...))
in
{
assert(i < t.length);
}
body
{
final switch(i)
{
foreach(ctI, m; t)
{
case ctI:
return t[ctI];
}
}
assert(0);
}
unittest
{
uint i = 2;
Tuple!(int, long, byte) myTuple;
myTuple[2] = 42;
auto a = myTuple.rtIdx(i);
static assert(is(typeof(a) == long));
assert(a == 42);
}
You could probably extend this to take expression tuples as well
or whatever the hell we call compiler tuples that contain runtime
values these days.
Alternatively, you could make rtIndex return a struct that
defines opIndex, so you could write
auto a = myTuple.rtIdx[i];
opSlice would be doable as well, to round it out.
Re: Run-time Indexing of a Compile-Time Tuple
On Tuesday, 12 May 2015 at 09:26:07 UTC, John Colvin wrote:
On Monday, 11 May 2015 at 22:46:00 UTC, Per Nordlöw wrote:
The pattern
final switch (_index)
{
import std.range: empty, front;
foreach (i, R; Rs)
{
case i:
assert(!source[i].empty);
return source[i].front;
}
}
occurring in roundRobin() and now also in my merge at
https://github.com/nordlow/justd/blob/master/range_ex.d#L604
is nor pretty nor concise.
Is there a better way of indexing a compile time tuple with a
run-time index?
It may have to work together with CommonType somehow as is
shown in my implementation of merge().
I wrote that code in roundRobin to replace a nightmare string
mixin. I can't see any way of getting around it, as there is no
meaningful CommonType for a tuple of arbitrary ranges. The body
of each case statement needs to know the index at compile-time.
Correction: there are ways around it, but none of them are
improvements.
Re: Weird OSX issue
On Monday, 27 April 2015 at 13:54:42 UTC, Steven Schveighoffer
wrote:
On 4/25/15 3:07 AM, Dan Olson wrote:
Jacob Carlborg writes:
On 2015-04-24 20:37, Steven Schveighoffer wrote:
So am I going crazy? Or is dmd doing things differently
depending on
where its environment is? Any compiler gurus out there
understand why
the symbol is different?
I don't want to file a bug with this, because it seems
dependent on
installation location, would possibly not be reproducible.
I can't reproduce this with DMD from DVM (compiler is
installed in the
user home directory).
I have lots of version laying around and they all worked fine
on the
posted code.
But maybe a clue here...
$ ~/dmd2.066.0/osx/bin/dmd mod1.d
$ nm mod1.o | grep start
U _D4core6thread6Thread5startMFZv
$ dmd mod1.d
$ nm mod1.o | grep start
U
_D4core6thread6Thread5startMFNbZC4core6thread6Thread
--- a/src/core/thread.d
+++ b/src/core/thread.d
@@ -587,7 +587,7 @@ class Thread
* Throws:
* ThreadException if the thread fails to start.
*/
-final Thread start()
+final Thread start() nothrow
in
{
assert( !next && !prev );
I wonder if dmd -v will show where its picking up stuff.
Thank you. It is something I missed. Lo and behold:
dmd -v mod1.d:
...
import
core.thread (/usr/share/dmd/src/druntime/import/core/thread.di)
...
~/Downloads/dmd2/osx/bin/dmd -v mod1.d:
...
importcore.thread
(/Users/steves/Downloads/dmd2/osx/bin/../../src/druntime/import/core/thread.d)
...
Hm... thread.di vs thread.d (and I didn't notice before because
all the other imports were .d, I just glanced over that detail).
And so, let's see here:
ls -l /usr/share/dmd/src/druntime/import/core/thread.*
-rw-rw-r-- 1 steves staff 157781 Mar 24 10:44 thread.d
-rw-rw-r--+ 1 steves staff 31382 Feb 24 2014 thread.di
Hey, looky there :)
So, looking at time machine (love that feature), I found that
when I installed 2.066, back in October, we had switched to
using thread.d instead of thread.di. But the installer did not
erase the old thread.di.
BTW, found this:
https://github.com/D-Programming-Language/druntime/pull/865
So I think the issue here is that the pkg installer on OSX does
not clean up the target directory if it already exists (or at
least purposely remove thread.di). Will look into fixing that.
At least now, it works properly, thanks (did a rm -rf
/usr/share/dmd and reinstall).
-Steve
Now that you mention it, I had similar issues on my Linux distro
at work (Ubuntu, alas!). The installer for 2.067.0 (or 2.066.0?)
didn't erase old files and I got compiler errors for no apparent
reason. What I did was to use dvm to install and manage my dmd
versions. This is much cleaner. I just didn't have the patience
to go through all the /dmd/ directories and clean them up.
dvm installs its own copy, it's not a system wide installation,
so you never have any issues with old files. I wouldn't want to
miss it now.
dvm install 2.067.1
later in your shell you just type
dvm use 2.067.1
https://github.com/jacob-carlborg/dvm
Re: Binding to C
On Tuesday, 12 May 2015 at 03:20:34 UTC, TJB wrote: I'm sure this question has been asked a thousand times. I've even asked similar questions in the past (I've been away for quite a while). But all of the tutorials that I have found assume quite a lot and leave a lot to be inferred. Is there a simple step by step tutorial demonstrating how to call C code from D? With the separate file contents and names and compilation instructions? Thanks, TJB Start with http://wiki.dlang.org/Bind_D_to_C http://dlang.org/interfaceToC.html http://dlang.org/htod.html http://wiki.dlang.org/Converting_C_.h_Files_to_D_Modules Best regards, Vadim
