Re: compile time regex
On Sat, Jun 16, 2012 at 10:50 PM, Dmitry Olshansky dmitry.o...@gmail.com wrote: On 17.06.2012 0:44, Ellery Newcomer wrote: There is a ctRegex; is there a compile time equivalent for match? No. Since last time I tried to hack through CTFE it failed horribly. Bleh, I'd rather first make sure that most regexes actually _compile_ at CTFE. Ellery, if you really need to match at CT, you can try Pegged: https://github.com/PhilippeSigaud/Pegged/wiki Be warned that while parsing at compile-time, it's not fast and will probably eat RAM like there is no tomorrow :) Youkei is developping CTPG, which is from the same family: https://github.com/youkei/ctpg/wiki/Home-en For both projects, the grammar is a PEG, not a regex, but the syntax is not that different. As PEG are more general, the engines are not optimized like std.regex. Philippe
Re: Getting started with threads in D
On Sun, 2012-06-17 at 03:15 +0200, Henrik Valter Vogelius Hansson wrote: Hi again! I have looked around a little with what D offers but don't know really what I should use since D offers several ways to use threads. Some more high level than others. Don't really also know which one would be suitable for me. My take on this is that as soon as an applications programmer talks about using threads in their program, they have admitted they are working at the wrong level. Applications programmers do not manage their control stacks, applications programmers do not manage their heaps, why on earth manage your threads. Threads are an implementation resource best managed by an abstraction. Using processes and message passing (over a thread pool, as you are heading towards in comments below) has proven over the last 30+ years to be the only scalable way of managing parallelism, so use it as a concurrency technique as well and get parallelism as near as for free as it is possible to get. Ancient models and techniques such as actors, dataflow, CSP, data parallelism are making a resurgence exactly because explicit shared memory multi-threading is an inappropriate technique. It has just taken the world 15+ years to appreciate this. A little background could help. I am a game developer and during my semester I want to experiment with making games in D. I use threads to separate some tasks that can easily work in parallel with each other. The most common being a Logic/Graphics separation. But as development progresses I usually add more threads like inside graphics I can end up with 2 or 3 more threads. I can only repeat the above: don't think in terms of threads and shared memory, think in terms of processes and messages passed between them. I want to avoid Amdahl's law as much as possible and have as small synchronization nodes. The data exchange should be as basic as possible but still have room for improvements and future additions. Isn't the current hypothesis that you can't avoid Amdahl's Law? If what you mean is that you want to ensure you have an embarrassingly parallel solution so that speed up is linear that seems entirely reasonable, but then D has a play in this game with the std.parallelism module. It uses the term task rather than process or thread to try and enforce an algorithm-focused view. cf. http://dlang.org/phobos/std_parallelism.html The Concurrency library looked very promising but felt like the synchronization wouldn't be that nice but it would provide a random-access to the data in your code. Correct me of course if I am wrong. Is there a good thread pool system that could be used? Does that system also handle solving dependencies in the work-flow? This is what we use at my work more or less. What makes you say synchronization is not that nice? Random access, data, threads and parallelism in the same paragraph raises a red flag of warning! std.concurrency is a realization of actors so there is effectively a variety of thread pool involved. std.parallelism has task pools explicitly. In worst case scenario I will just use the basic thread class and implement my own system above that. Then there is the question, is there any pitfalls in the current library that I should be aware of? I am sure D's current offerings are not perfect but they do represent a good part of the right direction to be travelling. What is missing is a module for dataflow processing(*) and one for CSP. Sadly I haven't had time to get stuck into doing an implementation as I had originally planned 18 months or so ago: most of my time is now in the Python and Groovy arena as that is where the income comes from. cf. GPars (http://gpars.codehaus.org) and Python-CSP – though the latter has stopped moving due to planning a whole new Python framework for concurrency and parallelism. (*) People who talk about you can implement dataflow with actors and vice versa miss the point about provision of appropriate abstractions with appropriate performance characteristics. -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder signature.asc Description: This is a digitally signed message part
readText fails to open file
This is kind of silly, and I probably missed something, but for
some reason I can't get any kind of text file opened when using
readText from std.file. This is what I'm trying to do:
import std.stdio;
import std.file;
int main(string[] args)
{
if (!isFile(args[0]))
{
writeln(Error: Input file does not exist or is not a
valid file!);
return 1;
}
auto LTFFile = chomp(readText(args[0]));
readln();
return 0;
}
Nice and simple, right? So I execute it:
./LTF2LIP LTF2LIP.d
std.utf.UTFException@std/utf.d(645): Invalid UTF-8 sequence (at
index 1)
And I'm sure that the file is in UTF-8, with LF line endings,
without a BOM. The same error is thrown when I try any other kind
of files. So what gives?..
Re: readText fails to open file
On 17.06.2012 12:21, GreatEmerald wrote:
This is kind of silly, and I probably missed something, but for some
reason I can't get any kind of text file opened when using readText from
std.file. This is what I'm trying to do:
import std.stdio;
import std.file;
int main(string[] args)
{
if (!isFile(args[0]))
{
writeln(Error: Input file does not exist or is not a valid file!);
return 1;
}
auto LTFFile = chomp(readText(args[0]));
readln();
return 0;
}
Nice and simple, right? So I execute it:
./LTF2LIP LTF2LIP.d
std.utf.UTFException@std/utf.d(645): Invalid UTF-8 sequence (at index 1)
It's a BOM most likely. There has been talk on making readText work with
them.
I'm not sure how you'd check there is no BOM, most text editors will
hide it. Hex editor may help though.
And I'm sure that the file is in UTF-8, with LF line endings, without a
BOM. The same error is thrown when I try any other kind of files. So
what gives?..
Other then this try cast(string)read(args[0]); it's not any worse.
--
Dmitry Olshansky
Re: readText fails to open file
On 06/17/2012 01:21 AM, GreatEmerald wrote:
This is kind of silly, and I probably missed something, but for some
reason I can't get any kind of text file opened when using readText from
std.file. This is what I'm trying to do:
import std.stdio;
import std.file;
int main(string[] args)
{
if (!isFile(args[0]))
{
writeln(Error: Input file does not exist or is not a valid file!);
return 1;
}
auto LTFFile = chomp(readText(args[0]));
readln();
return 0;
}
Nice and simple, right? So I execute it:
./LTF2LIP LTF2LIP.d
std.utf.UTFException@std/utf.d(645): Invalid UTF-8 sequence (at index 1)
And I'm sure that the file is in UTF-8, with LF line endings, without a
BOM. The same error is thrown when I try any other kind of files. So
what gives?..
Try args[1]. Your compiled program is surely not UTF-8. ;)
--
D Programming Language Tutorial: http://ddili.org/ders/d.en/index.html
Re: readText fails to open file
On Sunday, June 17, 2012 10:21:17 GreatEmerald wrote:
This is kind of silly, and I probably missed something, but for
some reason I can't get any kind of text file opened when using
readText from std.file. This is what I'm trying to do:
import std.stdio;
import std.file;
int main(string[] args)
{
if (!isFile(args[0]))
{
writeln(Error: Input file does not exist or is not a
valid file!);
return 1;
}
auto LTFFile = chomp(readText(args[0]));
readln();
return 0;
}
Nice and simple, right? So I execute it:
./LTF2LIP LTF2LIP.d
std.utf.UTFException@std/utf.d(645): Invalid UTF-8 sequence (at
index 1)
And I'm sure that the file is in UTF-8, with LF line endings,
without a BOM. The same error is thrown when I try any other kind
of files. So what gives?..
LOL. You're reading the wrong file. You have made the common mistake of
thinking that args[0] was the first argument that you passed to your program.
It's not. It's the name of your program. In this case, it would be
./LTF2LIP. It's args[1] which is LTF2LIP.d.
Also, you need to check exists before you check isFile, otherwise isFile will
blow up if the file doesn't exst. And both of those are properties, so you
should be calling them like
auto filename = args[1];
if(!filename.exists || !filename.isFile)
...
The -property flag enforces this behavior, and it will eventually become the
normal compiler behavior.
- Jonathan M Davis
Re: readText fails to open file
Oh, I just figured out what was going wrong. Apparently, args[0] is the path to the program itself, and not the first argument. args[1] is what I need to start reading from!
Re: sorting associative array's keys by values
well, the page I parse is automatically generated and thus allways contains /html. (if the page completely downloaded which it didn't). The second error I found (my mistake) is that newlines get scrambled up severely when downloading the page causing the markers I try to find to sometimes break down on two different lines. replacechars from std.string crashed too with the error invalid utf sequence. Now I use std.array's replace and everything (appears) to be working. If only I'd learned regex, I think that would've saved me quite some time.
Re: sorting associative array's keys by values
On Sunday, June 17, 2012 13:07:24 maarten van damme wrote: well, the page I parse is automatically generated and thus allways contains /html. That may be true, but if your code assumes that /html is there and it ever isn't for any reason, then you're going to get a RangeError thrown in non- release and undefined behavior in -release. It's generally a bad idea to assume that something is correct like that when you have zero control over it within your program, even if it's almost certainly correct. Something could go wrong, and it's better to throw an exception or do some other sort of error handling than it is to get an Error, or worse, undefined behavior. - Jonathan M Davis
Re: readText fails to open file
On Sunday, 17 June 2012 at 08:35:58 UTC, Jonathan M Davis wrote: Also, you need to check exists before you check isFile, otherwise isFile will blow up if the file doesn't exst. And both of those are properties, so you should be calling them like auto filename = args[1]; if(!filename.exists || !filename.isFile) ... The -property flag enforces this behavior, and it will eventually become the normal compiler behavior. - Jonathan M Davis Oh, I see, thanks!
Casting the Result of splitter() into a string array
A quick and simple question, but I couldn't find the answer to it by myself: how do I cast the return type of std.array.splitter() into a string[]? According to the compiler, splitter() has a return type Result. How useful... I assume it's a disguised tuple? If I try to simply assign the result to a string array, like this: string[] Bits = splitter( a b); I get an error Error: cannot implicitly convert expression (splitter( a b)) of type Result to string[]. Then if I make an explicit cast, I get Error: cannot cast from Result to string[]. So there must be something obvious that I'm missing... I also had the same problem when trying to cast the result of std.algorithm.filter!() to string[].
COM-Interfaces
Here http://dlang.org/interface.html#COM-Interfaces we have a link to Lionello Lunesu's article http://lunesu.com/uploads/ModernCOMProgramminginD.pdf Is it correct that he hasn't managed to convince Microsoft to release his sources and one of us has to rewrite his library if we want to use it? -- Денис В. Шеломовский Denis V. Shelomovskij
Re: Casting the Result of splitter() into a string array
On Sunday, June 17, 2012 13:39:19 GreatEmerald wrote: A quick and simple question, but I couldn't find the answer to it by myself: how do I cast the return type of std.array.splitter() into a string[]? According to the compiler, splitter() has a return type Result. How useful... I assume it's a disguised tuple? If I try to simply assign the result to a string array, like this: string[] Bits = splitter( a b); I get an error Error: cannot implicitly convert expression (splitter( a b)) of type Result to string[]. Then if I make an explicit cast, I get Error: cannot cast from Result to string[]. So there must be something obvious that I'm missing... I also had the same problem when trying to cast the result of std.algorithm.filter!() to string[]. They're ranges. This is probably the best explanation online at the moment: http://ddili.org/ders/d.en/ranges.html If you want to convert a range to an array, use std.array.array, though in the case of a string, to!string(range) will probably work. However, it makes no sense to convert the result of splitter to a string. What's the point of splitting them just to join them? splitter( a b ) is going to return a range of strings. If you were to do array(splitter(a b )), you'd get [, a, b]. - Jonathan M Davis
Re: Casting the Result of splitter() into a string array
On Sunday, 17 June 2012 at 11:58:58 UTC, Jonathan M Davis wrote: splitter( a b ) is going to return a range of strings. If you were to do array(splitter(a b )), you'd get [, a, b]. Aha, that worked! Thanks! Still makes me wonder how you're supposed to figure out that this type is a range and not anything else from the library documentation, though... Oh well.
Re: Casting the Result of splitter() into a string array
GreatEmerald: A quick and simple question, but I couldn't find the answer to it by myself: how do I cast the return type of std.array.splitter() into a string[]? splitter(), filter(), etc, are lazy, they return an iterable. Use split() instead of splitter(). Bye, bearophile
Re: Casting the Result of splitter() into a string array
On Sunday, June 17, 2012 14:11:27 GreatEmerald wrote: On Sunday, 17 June 2012 at 11:58:58 UTC, Jonathan M Davis wrote: splitter( a b ) is going to return a range of strings. If you were to do array(splitter(a b )), you'd get [, a, b]. Aha, that worked! Thanks! Still makes me wonder how you're supposed to figure out that this type is a range and not anything else from the library documentation, though... Oh well. The documentation says so in most cases. It certainly does for filter. It is true that it doesn't for spliter though. But pretty much everything in std.algorithm and std.range take and return ranges. And std.array generally takes arrays and returns either arrays or ranges. - Jonathan M Davis
Re: sorting associative array's keys by values
Ok, everything worked (nearly perfect). Sometimes the webpage gets completely messed up ( changes to d134 or something like that) but the tests handle it rather well. That's why I decided to improve it a bit and use treads. I've always been afraid of d treads because I never really got the grasp of that message passing system but hey, there is a first time for everything. Maybe I was a bit optimistic as I now already run into troubles. I have two arrays of the form int[string] To pass them from the spawned function back to the spawner I wanted to use tid.send( array1,array2); This was illegal, they have to be either immutable or shared. I've read somwhere that D tries to avoid shared where possible (with that message passing system) so I went for immutable and changed it over to tid.send(cast(immutable(int[string]))array1,...) (you get the idea) Not only does this look ugly, it's looks like what I'm doing isn't really correct. But anyway, onto the next part, receiving it. I used auto msg=receiveOnly!(int[string],int[string]); of course this fails because I send immutable items and want to receive mutables I then changed it to auto msg=receiveOnly!(immutable int[string],immutable int[string]); Compilation now fails with the very informative C:\D\dmd2\windows\bin\..\..\src\phobos\std\concurrency.d(612): Error: can only i nitialize const member _field_field_0 inside constructor C:\D\dmd2\windows\bin\..\..\src\phobos\std\concurrency.d(612): Error: can only i nitialize const member _field_field_1 inside constructor Why is treading made so damn hard? what have I done wrong now?
Re: Casting the Result of splitter() into a string array
If you want to convert a range to an array, use std.array.array This is a constant source of confusion and it also is a crappy design to use a function in a totally different module for this purpose imho. Can't these Result types get an eval() method and/or be made implicitly convertible to an array where appropriate? The C++ matrix library Eigen uses this approach and it works out fantastically. a*b returns an object representing this expression. If I do want to use this I write 'auto c=a*b;'. If I want the result I can explicitly use its eval() method (which is especially useful inside a bigger expression) or something like 'Matrix c=a*b;' which also evaluates the expression.
Re: sorting associative array's keys by values
another problem, when I do use shared.
My code is
int amountTreads = 20;
if(upperLimit-lowerLimitamountTreads)
amountTreads=upperLimit-lowerLimit;
int perTrade = (upperLimit - lowerLimit) / amountTreads;
for (int x = 0; x amountTreads; x++)
{
//we're at the last trade, parse all the rest
if (amountTreads - x == 1)
{
spawn(parse, lowerLimit + x * perTrade, upperLimit - x *
perTrade, thisTid);
}
else
{
spawn(parse, lowerLimit + x * perTrade, lowerLimit + (x +
1) * perTrade - 1, thisTid);
}
}
//wait for all those treads and process the response
for (int x = 0; x amountTreads; x++)
{
auto msg=receiveOnly!(shared int[string],shared int[string]);
int[string] tempDemands = cast(int[string])msg[0];
int[string] tempOffers = cast(int[string])msg[1];
foreach (string key; tempDemands.keys){
demands[key] += tempDemands[key];
}
foreach (string key; tempOffers.keys){
offers[key] += tempOffers[key];
}
writeln(ok, we're at ,x);
}
but when every tread has stopped working , that for-loop has only been
executed 17 times.
Re: Writing .di files
Am 17.06.2012 16:04, schrieb Minas: Is there a good tutorial about how to write .di files? You don't write them, the compiler outputs them with the -H option.
Re: Writing .di files
Al 17/06/12 16:04, En/na Minas ha escrit: Is there a good tutorial about how to write .di files? http://dlang.org/dmd-linux.html#interface_files -- Jordi Sayol
Simulating rectangular array
Has anyone come up with a template that can simulate a rectangular array and allow one to override opAssign to do special work? E.g.: Array!(2, 4) arr; arr[1][3] = foo; This would invoke opAssign in some templated struct Array. This would be super-easy to implement if I had an opIndex that could work with the above syntax. I know there's a [1, 3] syntax but I really need the array [1][3] syntax. Maybe Philippe has this in his templates book? (If not it would be a great addition to it)
Re: Simulating rectangular array
On Sunday, June 17, 2012 21:50:32 Andrej Mitrovic wrote: Has anyone come up with a template that can simulate a rectangular array and allow one to override opAssign to do special work? E.g.: Array!(2, 4) arr; arr[1][3] = foo; This would invoke opAssign in some templated struct Array. This would be super-easy to implement if I had an opIndex that could work with the above syntax. I know there's a [1, 3] syntax but I really need the array [1][3] syntax. Maybe Philippe has this in his templates book? (If not it would be a great addition to it) I believe that all you'd need to do is make it so that arr[1] returns an object which you can do the [3] on and get what you want. If returning an array works for that, then there you go. If you need another struct, then define the other struct. However, I would point out that you'd need a third template argument to Array for the type (unless you want it to hold a Variant). - Jonathan M Davis
Re: Simulating rectangular array
On Sun, Jun 17, 2012 at 9:50 PM, Andrej Mitrovic
andrej.mitrov...@gmail.com wrote:
Has anyone come up with a template that can simulate a rectangular
array and allow one to override opAssign to do special work? E.g.:
Array!(2, 4) arr;
arr[1][3] = foo;
This would invoke opAssign in some templated struct Array. This
would be super-easy to implement if I had an opIndex that could work
with the above syntax. I know there's a [1, 3] syntax but I really
need the array [1][3] syntax.
Maybe Philippe has this in his templates book? (If not it would be a
great addition to it)
No, I don't have something like this. Sorry if the question is stupid,
but how is your Array!(n,m) any different from string[n][m]?
And yes, Jonathan's idea is how I'd do it:
module test;
import std.conv;
import std.stdio;
class Array(T, size_t n,size_t m)
{
T[n*m] payload;
this(T[n*m] _payload) { payload = _payload;}
Indexer opIndex(size_t i)
{
assert(i n);
return new Indexer(i);
}
class Indexer
{
size_t i;
this(size_t _i) { i = _i;}
auto opIndex(size_t j)
{
assert(j m);
return payload[i*m+j];
}
}
}
void main()
{
auto arr = new Array!(string, 2,4)([abc, def, ghi, jkl,
mno, pqr, stu, vwx]);
writeln(arr[1][0]);
}
I guess this can extended to a general, N-dimensional, 'arrays'
(tensor, right?).
Re: COM-Interfaces
On Sunday, 17 June 2012 at 11:57:36 UTC, Denis Shelomovskij wrote: Here http://dlang.org/interface.html#COM-Interfaces we have a link to Lionello Lunesu's article http://lunesu.com/uploads/ModernCOMProgramminginD.pdf Is it correct that he hasn't managed to convince Microsoft to release his sources and one of us has to rewrite his library if we want to use it? Yes. You can look at Juno, it makes COM quite a bit easier. https://github.com/JesseKPhillips/Juno-Windows-Class-Library I've basically been trying to keep it compiling as D progresses so I don't have much in the way of how to use it, so you're stuck with the old documents on dsource http://dsource.org/projects/juno
