Re: most elegant functional way to make a histogram
On Saturday, 22 August 2015 at 02:12:41 UTC, BBasile wrote:
On Friday, 21 August 2015 at 20:09:22 UTC, Laeeth Isharc wrote:
I have four arrays of ints, each array representing a kind of
event associated with that int (they all map to the same
space). Each array might have the same number multiple times
and each array will be of different length.
So I would like to plot the int line on x axis and show number
of times that the number occurs for each array (4 bars for
each int).
It's easy to do with loops, but what's best
functional/algorithmic way, please? Brain tired today. I am
trying to use these opportunities to learn the algorithmic way
even if loop is more natural.
I could just make four new arrays of ints and use each instead
of a loop. Any better way?
Also, for bucketizing, any thoughts on best way to do using
phobos? (Cos probably I have too many ints and need to
bracket them to plot a histogram).
Sorry if this is unclear.
Laeeth.
loop-less approach, it consumes an InputRange in a recursive
function.
Assuming histogramData is a custom InputRange:
---
void upperLevel()
{
//histogramData = ...
proc(histogramData);
// continue once the range is consumed
}
void proc(ref histogramData)
{
//something with front
histogramData.popFront;
if (!histogramData.empty)
proc(histogramData);
}
---
I don't know if this approach can be used but this is an
alternative to loops.
Not necessarily the best because local variables in proc() can
lead to a stack overflow.
I believe in your example the compiler will perform TCO so you
should be okay.
Re: flush MessageBox
On Thursday, 20 August 2015 at 15:57:47 UTC, John Colvin wrote:
On Thursday, 20 August 2015 at 15:25:57 UTC, Chris wrote:
Is there a way to flush a thread's message box other than
aborting the thread? MailBox is private:
https://github.com/D-Programming-Language/phobos/blob/master/std/concurrency.d#L1778
flush from inside the thread? You could call receiveTimeout
with a 0 timer in a loop until it returns false.
Yes, from inside the thread. I have a worker thread (with
receiveTimeout) that is started when the program starts and sits
there waiting for input. I naively thought this was a great idea,
until I noticed that when input comes fast the mailbox grows and
the bloody thread won't stop until all the items in the mailbox
are processed. I was looking for something like `if (abort) {
mailbox.flush(); }`
Re: post on using go 1.5 and GC latency
Yes, Go has sacrificed some compute performance in favour of latency and convenience. They have also released GC improvement plans for 1.6: https://docs.google.com/document/d/1kBx98ulj5V5M9Zdeamy7v6ofZXX3yPziAf0V27A64Mo/edit It is rather obvious that a building a good concurrent GC is a time consuming effort.
controlling source directory of -cov
Currently, -cov requires you to run the coverage-instrumented program in the directory you compiled, so that it can find the source. Is there a way to tell the instrumented program where the source is if you run it in another directory? Alternatively, is there a way to specify the output directory for the list file? Here is the use case: On druntime, we are running some coverage tests, and we run them both in debug and release mode. However, on some systems these are run concurrently, and if they happen to be running the coverage analysis at the same time, they will conflict (both write to the same lst file). I tried running them from the build directory, but the output file is blank, because it cannot find the source file. At this point, I am working on a PR to symbolically link the source directory in the output directory, which will allow the generation to work, but I really don't like this solution. I'd much rather be able to control the output of the -cov report. If there isn't a way, I'll file an enhancement request for it. -Steve
Re: New to D - playing with Thread and false Sharing
On Thursday, 20 August 2015 at 15:31:13 UTC, tony288 wrote: So I wrong some code. But it seems the time to process a shared struct shared long is always the same. Regardless of adding paddings. Should it be different?
Re: flush MessageBox
On Friday, 21 August 2015 at 12:59:09 UTC, John Colvin wrote:
On Friday, 21 August 2015 at 10:43:22 UTC, Chris wrote:
On Thursday, 20 August 2015 at 15:57:47 UTC, John Colvin wrote:
On Thursday, 20 August 2015 at 15:25:57 UTC, Chris wrote:
Is there a way to flush a thread's message box other than
aborting the thread? MailBox is private:
https://github.com/D-Programming-Language/phobos/blob/master/std/concurrency.d#L1778
flush from inside the thread? You could call receiveTimeout
with a 0 timer in a loop until it returns false.
Yes, from inside the thread. I have a worker thread (with
receiveTimeout) that is started when the program starts and
sits there waiting for input. I naively thought this was a
great idea, until I noticed that when input comes fast the
mailbox grows and the bloody thread won't stop until all the
items in the mailbox are processed. I was looking for
something like `if (abort) { mailbox.flush(); }`
void flushMailbox()
{
bool r;
do
{
r = receiveTimeout(Duration.zero, (Variant _){});
} while(r);
}
You could optimise that to more efficiently deal with the
actual types you're receiving, instead of making a Variant
every time, but it's probably not worth it. The compiler might
even optimise it away anyway.
Wouldn't it be easier to have a library function that can empty
the mailbox immediately? It's a waste of time to have all items
in the mailbox crash against a wall, before you can go on as in
[
item1 // cancel
item2 === do nothing return;
item3 === do nothing return;
]
In parts I find std.concurrency lacking. Simple things sometimes
need workarounds.
Re: flush MessageBox
On Friday, 21 August 2015 at 10:43:22 UTC, Chris wrote:
On Thursday, 20 August 2015 at 15:57:47 UTC, John Colvin wrote:
On Thursday, 20 August 2015 at 15:25:57 UTC, Chris wrote:
Is there a way to flush a thread's message box other than
aborting the thread? MailBox is private:
https://github.com/D-Programming-Language/phobos/blob/master/std/concurrency.d#L1778
flush from inside the thread? You could call receiveTimeout
with a 0 timer in a loop until it returns false.
Yes, from inside the thread. I have a worker thread (with
receiveTimeout) that is started when the program starts and
sits there waiting for input. I naively thought this was a
great idea, until I noticed that when input comes fast the
mailbox grows and the bloody thread won't stop until all the
items in the mailbox are processed. I was looking for something
like `if (abort) { mailbox.flush(); }`
void flushMailbox()
{
bool r;
do
{
r = receiveTimeout(Duration.zero, (Variant _){});
} while(r);
}
You could optimise that to more efficiently deal with the actual
types you're receiving, instead of making a Variant every time,
but it's probably not worth it. The compiler might even optimise
it away anyway.
const-correct structs, best practices?
Is there a good posting somewhere that summarizes the current best practices for making const-correct (ie works for all of mutable/const/immutable) structs?
Re: New to D - playing with Thread and false Sharing
On Friday, 21 August 2015 at 12:45:52 UTC, Kagamin wrote: On Thursday, 20 August 2015 at 15:31:13 UTC, tony288 wrote: So I wrong some code. But it seems the time to process a shared struct shared long is always the same. Regardless of adding paddings. Should it be different? Hi all thanks, I guess there is a subtopic going on now. But I rewrote the code so that D Java looks fairly similar. First my mistake is I divided Nanosecond with too many zeros to get ms and hence Java seemed far superior and hence my frustration. That corrected D looked better - but I feel it is a bad comparison anyways different language etc... So won't say 1 is faster than the other. What I do find interesting is really now going back to the padding and false sharing. If I take the code here http://mechanical-sympathy.blogspot.co.uk/2011/07/false-sharing.html edit it as I mentioned above. The extra Padding Does affect quite a bit Java. However with D, not so much. So was curious if something is happening in the compiler. I remember I read a thread on this community about alignAllocate but couldn't really follow the post. So back to shared.. If Shared should be considered a normal accessible my multiple threads. Should I put this in somekind of lock/mutex/semaphore?
Re: const-correct structs, best practices?
On 08/21/2015 12:22 PM, Dicebot wrote: On Friday, 21 August 2015 at 15:00:04 UTC, Nick Sabalausky wrote: Is there a good posting somewhere that summarizes the current best practices for making const-correct (ie works for all of mutable/const/immutable) structs? - mark methods const - avoid reference type fields ;) What about inout? I have a struct that supports operator overloading, returning the struct's own type. After fiddling with that, I got a good laugh out of Meta's comment, and I think he may be right!
Best nogc method to insert element into the middle of an std.container Array?
I know inserting into the middle of arrays isn't the most efficient thing to do, but I have my reasons... I could increase the length by 1 and then shuffle all the values back one at a time... but I assume it'd be better to rewrite the back half as a single chunk? I don't really know how to get ranges to play nice with container arrays, and even if I can I don't know if they allocate in GC memory... Can anyone help me out with a good way to do this?
Re: const-correct structs, best practices?
On Friday, 21 August 2015 at 15:00:04 UTC, Nick Sabalausky wrote: Is there a good posting somewhere that summarizes the current best practices for making const-correct (ie works for all of mutable/const/immutable) structs? Prepare for pain.
Re: flush MessageBox
On Friday, 21 August 2015 at 14:35:53 UTC, Chris wrote:
On Friday, 21 August 2015 at 12:59:09 UTC, John Colvin wrote:
On Friday, 21 August 2015 at 10:43:22 UTC, Chris wrote:
On Thursday, 20 August 2015 at 15:57:47 UTC, John Colvin
wrote:
On Thursday, 20 August 2015 at 15:25:57 UTC, Chris wrote:
Is there a way to flush a thread's message box other than
aborting the thread? MailBox is private:
https://github.com/D-Programming-Language/phobos/blob/master/std/concurrency.d#L1778
flush from inside the thread? You could call receiveTimeout
with a 0 timer in a loop until it returns false.
Yes, from inside the thread. I have a worker thread (with
receiveTimeout) that is started when the program starts and
sits there waiting for input. I naively thought this was a
great idea, until I noticed that when input comes fast the
mailbox grows and the bloody thread won't stop until all the
items in the mailbox are processed. I was looking for
something like `if (abort) { mailbox.flush(); }`
void flushMailbox()
{
bool r;
do
{
r = receiveTimeout(Duration.zero, (Variant _){});
} while(r);
}
Thanks by the way. This does the trick.
Re: const-correct structs, best practices?
On Friday, 21 August 2015 at 15:00:04 UTC, Nick Sabalausky wrote: Is there a good posting somewhere that summarizes the current best practices for making const-correct (ie works for all of mutable/const/immutable) structs? - mark methods const - avoid reference type fields ;)
Re: const-correct structs, best practices?
On 8/21/15 12:31 PM, Nick Sabalausky wrote: On 08/21/2015 12:22 PM, Dicebot wrote: On Friday, 21 August 2015 at 15:00:04 UTC, Nick Sabalausky wrote: Is there a good posting somewhere that summarizes the current best practices for making const-correct (ie works for all of mutable/const/immutable) structs? - mark methods const - avoid reference type fields ;) What about inout? I have a struct that supports operator overloading, returning the struct's own type. After fiddling with that, I got a good laugh out of Meta's comment, and I think he may be right! inout should work when returning a piece of the struct. I found it incredibly easy when instrumenting dcollections. Just use it like you would const. Where you are going to run into problems is if you have things like ranges that need to partially cast to const. -Steve
most elegant functional way to make a histogram
I have four arrays of ints, each array representing a kind of event associated with that int (they all map to the same space). Each array might have the same number multiple times and each array will be of different length. So I would like to plot the int line on x axis and show number of times that the number occurs for each array (4 bars for each int). It's easy to do with loops, but what's best functional/algorithmic way, please? Brain tired today. I am trying to use these opportunities to learn the algorithmic way even if loop is more natural. I could just make four new arrays of ints and use each instead of a loop. Any better way? Also, for bucketizing, any thoughts on best way to do using phobos? (Cos probably I have too many ints and need to bracket them to plot a histogram). Sorry if this is unclear. Laeeth.
Re: most elegant functional way to make a histogram
On Friday, 21 August 2015 at 20:09:22 UTC, Laeeth Isharc wrote: I have four arrays of ints, each array representing a kind of event associated with that int (they all map to the same space). Each array might have the same number multiple times and each array will be of different length. So I would like to plot the int line on x axis and show number of times that the number occurs for each array (4 bars for each int). It's easy to do with loops, but what's best functional/algorithmic way, please? Brain tired today. I am trying to use these opportunities to learn the algorithmic way even if loop is more natural. I could just make four new arrays of ints and use each instead of a loop. Any better way? Also, for bucketizing, any thoughts on best way to do using phobos? (Cos probably I have too many ints and need to bracket them to plot a histogram). Sorry if this is unclear. I guess this kind of thing will do: upRangeHighs.each!((ref a)=(++histogram[a][0]));
Re: flush MessageBox
On Friday, 21 August 2015 at 14:35:53 UTC, Chris wrote: On Friday, 21 August 2015 at 12:59:09 UTC, John Colvin wrote: [...] Wouldn't it be easier to have a library function that can empty the mailbox immediately? It's a waste of time to have all items in the mailbox crash against a wall, before you can go on as in [ item1 // cancel item2 === do nothing return; item3 === do nothing return; ] In parts I find std.concurrency lacking. Simple things sometimes need workarounds. yes, it would be better. Please file an enhancement request: https://issues.dlang.org
Re: Best nogc method to insert element into the middle of an std.container Array?
On 08/21/2015 08:51 AM, Red Frog wrote:
I know inserting into the middle of arrays isn't the most efficient
thing to do, but I have my reasons... I could increase the length by 1
and then shuffle all the values back one at a time... but I assume it'd
be better to rewrite the back half as a single chunk?
I don't really know how to get ranges to play nice with container
arrays, and even if I can I don't know if they allocate in GC memory...
Can anyone help me out with a good way to do this?
only() and chain() come to mind. The followin does not produce an
array but you can call .array the whole thing later, e.g. in main()
after the @nogc function returns:
import std.algorithm;
import std.range;
@nogc auto use(int[] a, int[] b)
{
return chain(a, 42.only, b);
}
void main()
{
auto inserted = use([ 1, 2 ], [ 3, 4 ]);
assert(inserted.equal([ 1, 2, 42, 3, 4 ]));
}
Ali
Appender at CTFE?
Not at a pc, so can't test right now, but does Appender work at compile time? If not, does ~= still blow up CTFE memory usage like it used to? Any other best practice / trick for building strings in CTFE?
Re: Appender at CTFE?
On Friday, 21 August 2015 at 22:39:29 UTC, Nick Sabalausky wrote:
Not at a pc, so can't test right now, but does Appender work at
compile time? If not, does ~= still blow up CTFE memory usage
like it used to? Any other best practice / trick for building
strings in CTFE?
I did two experiments:
import std.conv;
import std.stdio;
import std.array;
import std.range;
import std.algorithm;
string f(T)(T strings) {
auto a = appender();
foreach (s ; strings)
a.put(s);
return a.data;
}
string g(T)(T strings) {
auto a = ;
foreach (s ; strings)
a ~= s;
return a;
}
void main(string[] args) {
enum a = iota(1).map!(to!string).f;
//enum a = iota(1).map!(to!string).g;
a.writeln;
}
Each make use of CTFE but the f (appender) variant blew my RAM
(old computer)
while the g (string) variant was nothing comparable in term of
memory
consumption and almost instantaneous. It looks like string
concatenation is
the best way to go, but this example is rather limited so
theoretical
confirmation would be great.
Re: most elegant functional way to make a histogram
On Friday, 21 August 2015 at 20:09:22 UTC, Laeeth Isharc wrote:
I have four arrays of ints, each array representing a kind of
event associated with that int (they all map to the same
space). Each array might have the same number multiple times
and each array will be of different length.
So I would like to plot the int line on x axis and show number
of times that the number occurs for each array (4 bars for each
int).
It's easy to do with loops, but what's best
functional/algorithmic way, please? Brain tired today. I am
trying to use these opportunities to learn the algorithmic way
even if loop is more natural.
I could just make four new arrays of ints and use each instead
of a loop. Any better way?
Also, for bucketizing, any thoughts on best way to do using
phobos? (Cos probably I have too many ints and need to bracket
them to plot a histogram).
Sorry if this is unclear.
Laeeth.
loop-less approach, it consumes an InputRange in a recursive
function.
Assuming histogramData is a custom InputRange:
---
void upperLevel()
{
//histogramData = ...
proc(histogramData);
// continue once the range is consumed
}
void proc(ref histogramData)
{
//something with front
histogramData.popFront;
if (!histogramData.empty)
proc(histogramData);
}
---
I don't know if this approach can be used but this is an
alternative to loops.
Not necessarily the best because local variables in proc() can
lead to a stack overflow.
Re: Appender at CTFE?
Thanks! I wouldn't have expected that about the memory. But I wonder how much of that memory usage with Appender was just all the template instantiations. I'll have to look into that when I get back.
Re: GC and MMM
On Thursday, 20 August 2015 at 17:13:33 UTC, Ilya Yaroshenko wrote: Hi All! Does GC scan manually allocated memory? Only if you ask GC to do it - by calling core.memory.addRange.
Re: Why does not my program is not running?
On Thursday, 20 August 2015 at 21:15:36 UTC, anonymous2 wrote: On Thursday, 20 August 2015 at 21:11:07 UTC, anonymous wrote: I severely limited the range of integer. I don't know off the top of my head how large you can make it without hitting overflow. I removed the file writing, because I'm not sure if you want to write it only when it doesn't exist, or only overwrite when it does exist. with integer == 66 the factorial overflows and becomes 0 (on my machine) = integer division by 0... The overflow happens at lower values, too. 66! has 2 64times as prime-factor, so a long becomes zero at that point. The overflow occures long ago (at 21!), but with 66! if becomes obvious, because you get a division by zero error.
main() return code.
Why is it acceptable to specify main as returning void (in addition to returning int)?
Re: main() return code.
On 08/21/2015 01:49 AM, Tony wrote: Why is it acceptable to specify main as returning void (in addition to returning int)? void in that context means automatically return 0 if main() exits without exception and non-zero if it exits with exception. Ali
post on using go 1.5 and GC latency
https://medium.com/@robin.verlangen/billions-of-request-per-day-meet-go-1-5-362bfefa0911 We then started analyzing the behavior of our Go application. On average the application spent ~ 2ms per request, which was great! It gave us 98 milliseconds to spare for network overhead, SSL handshake, DNS lookups and everything else that makes the internet work. Unfortunately the standard deviation of the latency was high, about 100 milliseconds. Meeting our SLA became a major gamble. With the “runtime” package of Go we started profiling the entire application and found out that garbage collection was the cause, resulting in 95-percentile latencies of 279 milliseconds… We decided to rewrite big chunks of the application to generate minimal or no garbage at all. This effectively helped reduce the interval at which garbage collection froze the rest of application to do its cleanup magic. But we were still having issues, so we decided to add more nodes to stay within our SLA. With over 80K requests per second at peak times, even minimal garbage can become a serious issue. ... Yesterday evening (19 August), the moment had finally arrived. A stable version 1.5 of Go was released, claiming: The “stop the world” phase of the collector will almost always be under 10 milliseconds and usually much less. Just a few hours after the release we rebuilt the application with the new version of Go 1.5 and ran our unit and functional tests; they all passed. It seemed too good to be true, so we put some effort in manually verifying the functionality. After a few hours we decided it was safe to release it to a single production node.
Re: New to D - playing with Thread and false Sharing
On Friday, 21 August 2015 at 02:44:50 UTC, Rikki Cattermole wrote: On 8/21/2015 3:37 AM, Dejan Lekic wrote: Keep in mind that in D everything is thread-local by default! :) For shared resources use __gshared or shared (although I do not know for sure whether shared works or not). Note: shared is __gshared but with mutex's added. Nope. shared is a type modifier that signifies the data is not guaranteed to be thread-local (and as such can be passed between threads). It generates no extra code and has no special ordering or atomicity guarantees. The way I look at it, shared is roughly equivalent to normal C/C++ variables.
