Re: std.concurrency.thisTid.send() feat. immutable(struct with several arrays)
27.02.2013 14:19, Jonathan M Davis пишет: On Wednesday, February 27, 2013 13:19:22 Alexandr Druzhinin wrote: This code doesn't compiles http://dpaste.dzfl.pl/706a0d12 But if you comment one of arrays it will do. I take a look at varaint.d but can't understand why two arrays cause this error. How can I send immutable struct that contains two or more arrays without any serializing? I don't exactly what's going on with your code here, but std.concurrency is very buggy with regards to what types it will and won't accept - including both immutable and arrays. These are just some of the reported issues: http://d.puremagic.com/issues/show_bug.cgi?id=6585 http://d.puremagic.com/issues/show_bug.cgi?id=5538 http://d.puremagic.com/issues/show_bug.cgi?id=8853 http://d.puremagic.com/issues/show_bug.cgi?id=9122 Your problem may be one of those. std.concurrency is fantastic as far as its basic design goes, but its implementation still needs a fair bit of work. It generally works, but it's fairly easy to find corner cases where it doesn't work. - Jonathan M Davis Thank you. I will learn to use reported issues.
Type of complex expression
Hi, Some time ago I asked how to efficiently parse a space delimited list of ints to array. Ireceived a good answer, but recently I discovered this: auto a= 1 2 3 4 5 .split( ) .filter!!a.empty .map!to!int(a); writeln(a); //writes [1, 2, 3, 4, 5] as expected. But: writeln(typeid(a).stringof); writes something interestingly crazy. I expected something like int[]. So of which type is that expression, and is this a good way to solve numbers parsing if I suppose that input is correct? Thank
Re: Type of complex expression
On Wednesday, February 27, 2013 11:02:59 Lubos Pintes wrote: Hi, Some time ago I asked how to efficiently parse a space delimited list of ints to array. Ireceived a good answer, but recently I discovered this: auto a= 1 2 3 4 5 .split( ) .filter!!a.empty .map!to!int(a); writeln(a); //writes [1, 2, 3, 4, 5] as expected. But: writeln(typeid(a).stringof); writes something interestingly crazy. I expected something like int[]. So of which type is that expression, and is this a good way to solve numbers parsing if I suppose that input is correct? The return type of map is most definitely _not_ an array. It's a range. writeln understands ranges and will print them out in the same way that it will print out arrays, which is why the output looks the same as if it were an array, but almost no range-based functions return arrays, because doing so would be inefficient (and in the case of infinite ranges, outright impossible). The exact type of a range depends on what returns it, and they're always templated types, so they also vary based of the function arguments. It's not really intended that you know or care what the range types are other than the fact that they're ranges and therefore have the API that ranges have. If you actually need to convert a range to an array, then use std.array.array, but range-based functions won't generally do that on their own, and most of the time, there's no reason to convert ranges to arrays. - Jonathan M Davis
Re: D timer
Am 26.02.2013 14:19, schrieb bearophile: David: Not sure what you mean, but I have a pretty solid Timer implementation (Threaded) https://github.com/Dav1dde/BraLa/blob/master/brala/utils/thread.d The file has no dependencies, so copy it over and have fun. License of the whole application is GPLv3, but if you need it relicensed drop me an e-mail or notify me somehow It's a common need. So maybe it's a good idea to put a version of that code in Phobos. Bye, bearophile I asked a while ago on the main NG if I should make it phobos compatible, answer was no.
Transparent ANSI to UTF-8 conversion
Hi, I would like to transparently convert from ANSI to UTF-8 when dealing with text files. For example here in Slovakia, virtually every text file is in Windows-1250. If someone opens a text file, he or she expects that it will work properly. So I suppose, that it is not feasible to tell someone if you want to use my program, please convert every text to UTF-8. To obtain the mapping from ANSI to Unicode for particular code page is trivial. Maybe even MultibyteToWidechar could help with this. I however need to know how to do it D-way. Could I define something like TextReader class? Or perhaps some support already exists somewhere? Thank
Re: Type of complex expression
Lubos Pintes: auto a= 1 2 3 4 5 .split( ) .filter!!a.empty .map!to!int(a); writeln(a); better (untested): auto a = 1 2 3 4 5 .split .map!(to!int) .writeln; Bye, bearophile
Re: A little of coordination for Rosettacode
On 2013-02-26 15:10, bearophile wrote: This third version is much simpler and it seems good enough for Rosettacode: http://codepad.org/YJjb1t91 Nice. Myself I'd add bits.reverse; after the N.iota.map..., because I'm more used to a truth table where the left-most operands iterate the slowest.
Re: A little of coordination for Rosettacode
On 2013-02-27 12:47, FG wrote: On 2013-02-26 15:10, bearophile wrote: This third version is much simpler and it seems good enough for Rosettacode: http://codepad.org/YJjb1t91 Nice. Myself I'd add bits.reverse; after the N.iota.map..., because I'm more used to a truth table where the left-most operands iterate the slowest. or, without reverse, but perhaps not very clear: N.iota.map!(j = !!(i (1 (N - j - 1.copy(bits[]);
Re: Transparent ANSI to UTF-8 conversion
On Wednesday, 27 February 2013 at 10:56:16 UTC, Lubos Pintes wrote: Hi, I would like to transparently convert from ANSI to UTF-8 when dealing with text files. For example here in Slovakia, virtually every text file is in Windows-1250. If someone opens a text file, he or she expects that it will work properly. So I suppose, that it is not feasible to tell someone if you want to use my program, please convert every text to UTF-8. To obtain the mapping from ANSI to Unicode for particular code page is trivial. Maybe even MultibyteToWidechar could help with this. I however need to know how to do it D-way. Could I define something like TextReader class? Or perhaps some support already exists somewhere? Thank I'd say the D way would be to simply exploit the fact that UTF is built into the language, and as such, not worry about encoding, and use raw code points. You get you Codepage to unicode *codepoint* table, and then you simply map each character to a dchar. From there, D will itself convert your raw unicode (aka UTF-32) to UTF8 on the fly, when you need it. For example, writing to a file will automatically convert input to UTF-8. You can also simply use std.conv.to!string to convert any UTF scheme to UTF-8 (or any other UTF too for that matter). This may not be as efficient as a true codepage to UTF8 table but: 1) Given you'll most probably be IO bound anyways, who cares? 2) Scalability. D does everything but the code page to code point mapping. Why bother doing any more than that?
Linking C and D
I'm trying to get the hello world of cross compiling working:
main.d
//
extern (C) void foo();
void main()
{
foo();
}
//
foo.c
//
#include stdio.h
void foo()
{
printf(hello world);
}
//
I can't seem to get the executable to link correctly.
I'm using gcc and dmd on windows.
I'm building foo.c with:
gcc -c foo.c -o foo.obj
Then I build my exe with:
dmd foo.obj main.d
But I get:
OPTLINK (R) for Win32 Release 8.00.12
Copyright (C) Digital Mars 1989-2010 All rights reserved.
http://www.digitalmars.com/ctg/optlink.html
test.obj Offset 0H Record Type 004C
Error 138: Module or Dictionary corrupt
--- errorlevel 1
...Help ?
Re: Linking C and D
On Wednesday, 27 February 2013 at 16:12:13 UTC, monarch_dodra wrote: I'm trying to get the hello world of cross compiling working: The short story is you can't link GCC and DMD object files on win32 because DMD emits OMF, GCC emits COFF, these are incompatible. You might want to read this: http://www.gamedev.net/blog/1140/entry-2254003-binding-d-to-c/ But I get: OPTLINK (R) for Win32 Release 8.00.12 Copyright (C) Digital Mars 1989-2010 All rights reserved. http://www.digitalmars.com/ctg/optlink.html test.obj Offset 0H Record Type 004C Error 138: Module or Dictionary corrupt --- errorlevel 1 I think we should try to implement a check in Optlink so it errors with a nicer cannot link objects files in COFF format rather than what it does right now.
Re: Linking C and D
On Wednesday, 27 February 2013 at 16:37:06 UTC, Alexandr Druzhinin wrote: 27.02.2013 23:12, monarch_dodra пишет: I can't seem to get the executable to link correctly. I'm using gcc and dmd on windows. I'm building foo.c with: gcc -c foo.c -o foo.obj Then I build my exe with: dmd foo.obj main.d But I get: OPTLINK (R) for Win32 Release 8.00.12 Copyright (C) Digital Mars 1989-2010 All rights reserved. http://www.digitalmars.com/ctg/optlink.html test.obj Offset 0H Record Type 004C Error 138: Module or Dictionary corrupt --- errorlevel 1 ...Help ? Under Windows I use dmc + dmd and gdc + gcc - all these combinations work fine. But I only tested it on simple and small files. There is also unilink (ftp://ftp.styx.cabel.net/pub/UniLink/) - they say it can link together omf and coff, but I don't try it TY. dmc works. Unfortunatly, I'm actually doing this to compile font files into my D project. They can reach 2_000_000 lines, and weight in at over 100Mo of code. dmc seems to choke on these: // nbytes = 102000, ph_maxsize = 65520 Internal error: ph.c 1854 // dmc seems to start choking when my font files start to reach about 15 Mo. I suppose there's a switch somewhere, but I've never used dmc before, so all the switches are unknown to me. I guess I'll just have to learn (anybody know?). If worst comes to worst, I can try to split the font files, but since they are tool generated, I really shouldn't be touching them.
Re: Linking C and D
On Wednesday, 27 February 2013 at 16:20:43 UTC, Andrej Mitrovic wrote: On Wednesday, 27 February 2013 at 16:12:13 UTC, monarch_dodra wrote: I'm trying to get the hello world of cross compiling working: The short story is you can't link GCC and DMD object files on win32 because DMD emits OMF, GCC emits COFF, these are incompatible. You might want to read this: http://www.gamedev.net/blog/1140/entry-2254003-binding-d-to-c/ But I get: OPTLINK (R) for Win32 Release 8.00.12 Copyright (C) Digital Mars 1989-2010 All rights reserved. http://www.digitalmars.com/ctg/optlink.html test.obj Offset 0H Record Type 004C Error 138: Module or Dictionary corrupt --- errorlevel 1 I think we should try to implement a check in Optlink so it errors with a nicer cannot link objects files in COFF format rather than what it does right now. Thanks.
Re: Linking C and D
27.02.2013 23:12, monarch_dodra пишет: I can't seem to get the executable to link correctly. I'm using gcc and dmd on windows. I'm building foo.c with: gcc -c foo.c -o foo.obj Then I build my exe with: dmd foo.obj main.d But I get: OPTLINK (R) for Win32 Release 8.00.12 Copyright (C) Digital Mars 1989-2010 All rights reserved. http://www.digitalmars.com/ctg/optlink.html test.obj Offset 0H Record Type 004C Error 138: Module or Dictionary corrupt --- errorlevel 1 ...Help ? Under Windows I use dmc + dmd and gdc + gcc - all these combinations work fine. But I only tested it on simple and small files. There is also unilink (ftp://ftp.styx.cabel.net/pub/UniLink/) - they say it can link together omf and coff, but I don't try it
Re: Linking C and D
27.02.2013 23:50, monarch_dodra пишет: dmc seems to start choking when my font files start to reach about 15 Mo. I suppose there's a switch somewhere, but I've never used dmc before, so all the switches are unknown to me. I guess I'll just have to learn (anybody know?). If worst comes to worst, I can try to split the font files, but since they are tool generated, I really shouldn't be touching them. IIRC, there is such switch, but I can't remember it, may be http://www.digitalmars.com/ctg/sc.html#dashCapHCapP ?
Re: Template alias parameter does not accept functions
Actually it looks like a somewhat broken behavior as it works for parameter-less functions.
Re: Transparent ANSI to UTF-8 conversion
27-Feb-2013 16:20, monarch_dodra пишет: On Wednesday, 27 February 2013 at 10:56:16 UTC, Lubos Pintes wrote: Hi, I would like to transparently convert from ANSI to UTF-8 when dealing with text files. For example here in Slovakia, virtually every text file is in Windows-1250. If someone opens a text file, he or she expects that it will work properly. So I suppose, that it is not feasible to tell someone if you want to use my program, please convert every text to UTF-8. To obtain the mapping from ANSI to Unicode for particular code page is trivial. Maybe even MultibyteToWidechar could help with this. I however need to know how to do it D-way. Could I define something like TextReader class? Or perhaps some support already exists somewhere? Thank I'd say the D way would be to simply exploit the fact that UTF is built into the language, and as such, not worry about encoding, and use raw code points. You get you Codepage to unicode *codepoint* table, and then you simply map each character to a dchar. From there, D will itself convert your raw unicode (aka UTF-32) to UTF8 on the fly, when you need it. For example, writing to a file will automatically convert input to UTF-8. You can also simply use std.conv.to!string to convert any UTF scheme to UTF-8 (or any other UTF too for that matter). Making a table that translates ANSI to UTF8 is trivially constructible using CTFE from the static one that does ANSI - dchar. This may not be as efficient as a true codepage to UTF8 table but: 1) Given you'll most probably be IO bound anyways, who cares? With in-memory transcoding you won't be. Text editors are typically all in-memory or mmap-ed. 2) Scalability. D does everything but the code page to code point mapping. Why bother doing any more than that? -- Dmitry Olshansky
Re: Template alias parameter does not accept functions
On 27/02/2013 15:20, Dicebot wrote:
Looking here: http://dlang.org/template.html#TemplateAliasParameter
There are no function symbols in the list. And quick check that it won't
compile.
Two questions arise:
a) Why so?
b) Is there a workaround?
This has sometimes come up in the NG, with some support for the feature.
See also:
http://d.puremagic.com/issues/show_bug.cgi?id=7308
bearophile shows the workaround - use another template:
template ID(alias a){alias a ID;}
void main(){
alias ID!(x = x ^^ 2) sqrTemplate;
}
On 27/02/2013 17:17, Dicebot wrote:
Actually it looks like a somewhat broken behavior as it works for
parameter-less functions.
I'm not sure what you mean, can you show the code it works for?
Re: Template alias parameter does not accept functions
On 27/02/2013 17:38, Nick Treleaven wrote: On 27/02/2013 15:20, Dicebot wrote: Looking here: http://dlang.org/template.html#TemplateAliasParameter There are no function symbols in the list. And quick check that it won't compile. Two questions arise: a) Why so? b) Is there a workaround? Actually I may have misunderstood your question. Anyway, please show the code that doesn't work.
Re: Template alias parameter does not accept functions
Ye, lambda litteral issue is different one. My code example: http://dpaste.1azy.net/b06370ea
Re: scope(exit) stack = double free or corruption (fasttop) ... help?
On 02/26/2013 04:24 PM, Ben Davis wrote:
On 26/02/2013 06:21, Charles Hixson wrote:
On 02/24/2013 05:39 PM, H. S. Teoh wrote:
On Sun, Feb 24, 2013 at 03:14:01PM -0800, Charles Hixson wrote:
Given a struct with:
~this()
{ close(); }
void close()
{ if (currentKey !is null) currentKey = null;
if (cursor is null) return;
tcbdbcurdel(cursor);
}
and:
scope (exit) if (bdb !is null) tcbdbclose(bdb);
//scope(exit) cur.close; //- cur is the struct noted above
[...]
The struct dtor is automatically called upon exiting the scope, so your
scope(exit) here is redundant, and is the cause of the double free.
T
Sorry, but that wasn't the answer...which was trivial, when I saw it. I
needed to change the close() routine to:
void close()
{ if (currentKey !is null) currentKey = null;
if (cursor is null) return;
tcbdbcurdel(cursor);
cursor = null;
}
Apparently the library didn't null the cursor after it closed
(deleted) it.
You're both right.
Your struct is presumably declared as a simple local variable, like this:
someFunction() {
YOUR_STRUCT cur;
...
}
What we're saying is that cur's destructor is called automatically as
soon as execution reaches the }, even if an exception is thrown or a
break/continue/return/goto jumps out.
So when you write scope (exit) cur.close();, you're queueing the
close() to happen under the same circumstances. This doesn't stop the
destructor call happening. Your destructor calls close() too, so it gets
called twice.
You should also know that the library you're using *can't* set 'cursor'
to null for you, because you're in control of the memory location where
'cursor' is stored (it's inside your struct), and when you pass it to
tcbdbcurdel(), you're only passing a copy of the value. In order for it
to set it to null for you, you would have pass a pointer to the value,
by writing 'cursor'. (This is not strictly true for D or C++ functions
since they can take 'references' which are implicit pointers, but you
said it's a C library - and in any case, 'ref' parameters will usually
be much more obviously 'ref' parameters than this one, if the API is
well designed.)
So - setting cursor to null is a good safe fix, as it makes it safe to
call close() more than once - but you also don't need the 'scope
(exit)', as you can rely on the destructor being called.
(But further to that - you can only rely on the destructor being called
if it's a struct (and not a pointer to one), or a scope class (a class
with the 'scope' attribute). Normal classes will be destroyed eventually
by the GC, but not at a well-defined time, and there's (probably?) no
guarantee it'll be called before the program exits.)
I guess D isn't as simple as it wanted to be! But it is powerful :)
I want the design to work whether the struct is allocated on the stack
or on the heap, so while in this particular instance it was on the
stack, I don't want a design that assumes that. After I get it working
right I intend to wrap everything up into probably two structs. So I
*can't* rely on the destructor being called...not in a timely way. I'm
using scope(exit) because try{}finally{} would be a nuisance...but the
database must ensure that the cursor is properly closed. (At least I'm
assuming so.)
Still, you were right that the library couldn't set my local pointer to
null. So I made *another* mistake. *grrr*
Re: Template alias parameter does not accept functions
Fuck. Optional. Parens. Thank you. I have tried some other code than .stringof but guess it invoked Symbols somewhere silently, too.
Re: Template alias parameter does not accept functions
On 27/02/2013 18:06, Dicebot wrote:
Ye, lambda litteral issue is different one.
OK.
My code example: http://dpaste.1azy.net/b06370ea
The problem may be related to optional parentheses and stringof, and may
be a compiler bug.
You can pass function names as a template alias parameter:
import std.stdio;
int f(int i) {return i^^2;}
template check(alias Symbol)
{
enum check = Symbol(5); // CTFE
}
void main()
{
//~ pragma(msg, f.stringof); //error
writeln(check!f); // 25
}
Parsing string to string?
I have a text file, that contains text with escaped characters, eg: // hello\tworld. this line\ncontains a break. and this one a\x20binary character. and this one\u0020an escaped unicode. // The idea is that once parse line by line, I want 4 strings (1 for each line), but with the escaped characters parsed to their normal value. I'm having trouble doing this efficiently. I can do it with std.conv.parse, but provided I preppend and append a double quote to my strings first. It's not the most efficient way to do it, but it works. It's kind of hackish though :/ I was wondering if there anything in phobos that could do this more naturally? Or in an idiomatic fashion?
Re: Linking C and D
On Wednesday, 27 February 2013 at 17:08:38 UTC, Alexandr Druzhinin wrote: 27.02.2013 23:50, monarch_dodra пишет: dmc seems to start choking when my font files start to reach about 15 Mo. I suppose there's a switch somewhere, but I've never used dmc before, so all the switches are unknown to me. I guess I'll just have to learn (anybody know?). If worst comes to worst, I can try to split the font files, but since they are tool generated, I really shouldn't be touching them. IIRC, there is such switch, but I can't remember it, may be http://www.digitalmars.com/ctg/sc.html#dashCapHCapP ? I found a couple posts about that, but it doesn't seem to do much. I'll just have to try harder.
Re: Linking C and D
With Dmd you can use dmc directly. Second way is: you can compile your code as dll library and then create import library with implib utility for win32, for win64 a import library can be used directly. First way works good, but I prefer second one. Both approaches for win32 and win64 works good enough and stable.
Re: Linking C and D
Also C runtime can be statically compiled with your c/dll library. You just need copy c dll and your d exe without additional dependencies. C dll can be prepared by visual c.
nWayUnion(tuple)?
Currently std.algorithm.nWayUnion requires an array of ranges,
because it internally keeps them in a heap, to be fast when you
give it hundreds+ of ranges.
But in some cases I'd like to merge different types of ranges,
that I can't put in an array. Is this use case worth supporting
(usually the number of ranges is small so for such use cases a
heap is not so needed)?
import std.algorithm: nWayUnion, map;
import std.range: iota;
import std.typecons: tuple;
void main() {
auto a = iota(10);
auto b = [3, 6, 9];
auto c = iota(11).map!q{a * a};
auto r = nWayUnion(tuple(a, b, c));
}
(Or maybe I am missing something, and this is already possible in
Phobos. This happened some times in past because it's not easy to
fully understand the high flexibility of std.algorithm).
Bye,
bearophile
Re: Linking C and D
On Wednesday, 27 February 2013 at 18:40:40 UTC, monarch_dodra wrote: On Wednesday, 27 February 2013 at 17:08:38 UTC, Alexandr Druzhinin wrote: 27.02.2013 23:50, monarch_dodra пишет: dmc seems to start choking when my font files start to reach about 15 Mo. I suppose there's a switch somewhere, but I've never used dmc before, so all the switches are unknown to me. I guess I'll just have to learn (anybody know?). If worst comes to worst, I can try to split the font files, but since they are tool generated, I really shouldn't be touching them. IIRC, there is such switch, but I can't remember it, may be http://www.digitalmars.com/ctg/sc.html#dashCapHCapP ? I found a couple posts about that, but it doesn't seem to do much. I'll just have to try harder. You could try using GDC and GCC instead. There are MinGW GDC builds at https://bitbucket.org/goshawk/gdc/downloads .
Re: Transparent ANSI to UTF-8 conversion
I don't understand the CTFE usage in this context. I thought about something like dchar[] windows_1250=[...]; Isn't this enough? Thank Dňa 27. 2. 2013 18:32 Dmitry Olshansky wrote / napísal(a): 27-Feb-2013 16:20, monarch_dodra пишет: On Wednesday, 27 February 2013 at 10:56:16 UTC, Lubos Pintes wrote: Hi, I would like to transparently convert from ANSI to UTF-8 when dealing with text files. For example here in Slovakia, virtually every text file is in Windows-1250. If someone opens a text file, he or she expects that it will work properly. So I suppose, that it is not feasible to tell someone if you want to use my program, please convert every text to UTF-8. To obtain the mapping from ANSI to Unicode for particular code page is trivial. Maybe even MultibyteToWidechar could help with this. I however need to know how to do it D-way. Could I define something like TextReader class? Or perhaps some support already exists somewhere? Thank I'd say the D way would be to simply exploit the fact that UTF is built into the language, and as such, not worry about encoding, and use raw code points. You get you Codepage to unicode *codepoint* table, and then you simply map each character to a dchar. From there, D will itself convert your raw unicode (aka UTF-32) to UTF8 on the fly, when you need it. For example, writing to a file will automatically convert input to UTF-8. You can also simply use std.conv.to!string to convert any UTF scheme to UTF-8 (or any other UTF too for that matter). Making a table that translates ANSI to UTF8 is trivially constructible using CTFE from the static one that does ANSI - dchar. This may not be as efficient as a true codepage to UTF8 table but: 1) Given you'll most probably be IO bound anyways, who cares? With in-memory transcoding you won't be. Text editors are typically all in-memory or mmap-ed. 2) Scalability. D does everything but the code page to code point mapping. Why bother doing any more than that?
Re: Parsing string to string?
And what about the opposite way? I like how Python is representing strings. Good when you want to partially inspect something binary. Dňa 27. 2. 2013 19:39 monarch_dodra wrote / napísal(a): I have a text file, that contains text with escaped characters, eg: // hello\tworld. this line\ncontains a break. and this one a\x20binary character. and this one\u0020an escaped unicode. // The idea is that once parse line by line, I want 4 strings (1 for each line), but with the escaped characters parsed to their normal value. I'm having trouble doing this efficiently. I can do it with std.conv.parse, but provided I preppend and append a double quote to my strings first. It's not the most efficient way to do it, but it works. It's kind of hackish though :/ I was wondering if there anything in phobos that could do this more naturally? Or in an idiomatic fashion?
Re: Transparent ANSI to UTF-8 conversion
28-Feb-2013 00:35, Lubos Pintes пишет: I don't understand the CTFE usage in this context. I thought about something like dchar[] windows_1250=[...]; Isn't this enough? Thank It's fine. What I've meant is if all you want to do is convert ANSI - UTF8 there is no need to convert to dchar and then to UTF-8 chars. so the table becomes more like: char[][] windows_1250_to_UTF8 = [...]; Or rather (far better memory footprint): char[2][] windows_1250_to_UTF8 = [ ... ]; I think 2 UTF-8 chars should be enough for your codepage. Then CTFE is just a tool create one table from another : char[2][] windows_1250UTF = createUTF8Table(windows_1250); The point is that inside of createUTF8Table you create an array it by using new and simple loops + std.utf.encode just like in normal code but it'll be CTFE-ed. Same goes for going backwards - you can treat char[2] as ushort and do the tables. Though now it may have gaps due to encoding not being linear but rather having some stride with certain period. Dňa 27. 2. 2013 18:32 Dmitry Olshansky wrote / napísal(a): 27-Feb-2013 16:20, monarch_dodra пишет: On Wednesday, 27 February 2013 at 10:56:16 UTC, Lubos Pintes wrote: Hi, I would like to transparently convert from ANSI to UTF-8 when dealing with text files. For example here in Slovakia, virtually every text file is in Windows-1250. If someone opens a text file, he or she expects that it will work properly. So I suppose, that it is not feasible to tell someone if you want to use my program, please convert every text to UTF-8. To obtain the mapping from ANSI to Unicode for particular code page is trivial. Maybe even MultibyteToWidechar could help with this. I however need to know how to do it D-way. Could I define something like TextReader class? Or perhaps some support already exists somewhere? Thank I'd say the D way would be to simply exploit the fact that UTF is built into the language, and as such, not worry about encoding, and use raw code points. You get you Codepage to unicode *codepoint* table, and then you simply map each character to a dchar. From there, D will itself convert your raw unicode (aka UTF-32) to UTF8 on the fly, when you need it. For example, writing to a file will automatically convert input to UTF-8. You can also simply use std.conv.to!string to convert any UTF scheme to UTF-8 (or any other UTF too for that matter). Making a table that translates ANSI to UTF8 is trivially constructible using CTFE from the static one that does ANSI - dchar. This may not be as efficient as a true codepage to UTF8 table but: 1) Given you'll most probably be IO bound anyways, who cares? With in-memory transcoding you won't be. Text editors are typically all in-memory or mmap-ed. 2) Scalability. D does everything but the code page to code point mapping. Why bother doing any more than that? -- Dmitry Olshansky
Re: nWayUnion(tuple)?
On 02/27/2013 11:36 AM, bearophile wrote:
But in some cases I'd like to merge different types of ranges, that I
can't put in an array. Is this use case worth supporting (usually the
number of ranges is small so for such use cases a heap is not so needed)?
nWayUnion requires a range of ranges. The simplest thing to do for that
is to pass an array. It is not very pretty, but to match the elements of
that array, there is inputRangeObject:
import std.algorithm;
import std.range;
import std.stdio;
void main() {
InputRange!int a = inputRangeObject(iota(10));
InputRange!int b = inputRangeObject([3, 6, 9]);
InputRange!int c = inputRangeObject(iota(11).map!q{a * a});
auto r = nWayUnion([ a, b, c ]);
writeln(r);
}
The output:
[0, 0, 1, 1, 2, 3, 3, 4, 4, 5, 6, 6, 7, 8, 9, 9, 9, 16, 25, 36, 49, 64,
81, 100]
Phobos could have a function so that the code could be cleaner. A
function that supports treat these as InputRanges of ints:
nWayUnion(inputRanges(a, b, c));
But it is not there. :)
Ali
Best way to handle settings files (ini file syntax or similar)
Hi, I was wondering what would be the most straightforward way to handle settings files in D, currently? Is there support for ini filesor something similar, or would I be better off going with JSON or XML or similar? BR // Samuel
Re: Best way to handle settings files (ini file syntax or similar)
I use this simple snippet to get quick and dirty key-value config:
---
string[string] data;
foreach( line; readText(filename).splitLines() )
{
auto config_pair = array(
filter!(a.length 0)(
map!(strip)(
line.splitter(=)
)
)
);
data[config_pair[0]] = config_pair[1];
}
---
For anything even remotely complex I would have probably chosen
JSON, either new std.json pending for review (not current
std.json!) or vibe.data.json from vibed.org project.
Re: Best way to handle settings files (ini file syntax or similar)
Btw now I have actually noticed it is really bad and dirty, so good to sometimes check your 2-year code. I think you can write better one anyway, just an example of how small it can be.
Re: Parsing string to string?
On Wednesday, 27 February 2013 at 20:37:01 UTC, Lubos Pintes wrote: And what about the opposite way? I like how Python is representing strings. Good when you want to partially inspect something binary. Dňa 27. 2. 2013 19:39 monarch_dodra wrote / napísal(a): I have a text file, that contains text with escaped characters, eg: // hello\tworld. this line\ncontains a break. and this one a\x20binary character. and this one\u0020an escaped unicode. // The idea is that once parse line by line, I want 4 strings (1 for each line), but with the escaped characters parsed to their normal value. I'm having trouble doing this efficiently. I can do it with std.conv.parse, but provided I preppend and append a double quote to my strings first. It's not the most efficient way to do it, but it works. It's kind of hackish though :/ I was wondering if there anything in phobos that could do this more naturally? Or in an idiomatic fashion? using formattedWrite, I can print it out the same way, but there will also be the trailing I'll have to chop off manually (hence why I have to pass through a formatted write first, as opposed to a straight up writef).
Playing with ranges and ufcs
This command: writeln(iota(10).cycle()[5..2].take(4)); print: [5, 6, 7, 8] Shouldn't [5..2] slice throw a compile/runtime error?
Re: Playing with ranges and ufcs
Andrea Fontana:
writeln(iota(10).cycle()[5..2].take(4));
print:
[5, 6, 7, 8]
Shouldn't [5..2] slice throw a compile/runtime error?
Please file it in bugzilla.
The opSlice of cycle() lacks those pre-conditions or tests, and
there are not enough unittests in Phobos to catch this simple bug:
auto opSlice(size_t i, size_t j)
{
auto retval = this.save;
retval._index += i;
return takeExactly(retval, j - i);
}
j - i is positive because those numbers are unsigned, and because
D lacks a run-time errors for integral values because the stupid
Walter thinks those run-time are too much slow (it's not actually
true. Even Clang designers have understood it), and Don thinks
that Everything is Fine in D:
import std.stdio;
struct Foo {
auto opSlice(size_t i, size_t j) {
writeln(j - i); // Prints: 4294967293
}
}
void main() {
Foo f;
f[5 .. 2];
}
Maybe we need to run something similar to QuickCheck
(http://en.wikipedia.org/wiki/QuickCheck ) on Phobos.
Bye,
bearophile
Re: nWayUnion(tuple)?
Ali Çehreli: It is not very pretty, but to match the elements of that array, there is inputRangeObject: OK. Phobos could have a function so that the code could be cleaner. A function that supports treat these as InputRanges of ints: nWayUnion(inputRanges(a, b, c)); But it is not there. :) If we add an overload of nWayUnion (better named nWayMerge: http://d.puremagic.com/issues/show_bug.cgi?id=6718 ) then there's no need to use inputRangeObject... The question is how much common my use case (mixed type iterables) is. Thank you, bearophile
Re: nWayUnion(tuple)?
On 02/27/2013 03:02 PM, bearophile wrote:
If we add an overload of nWayUnion (better named nWayMerge:
http://d.puremagic.com/issues/show_bug.cgi?id=6718 ) then there's no
need to use inputRangeObject...
The question is how much common my use case (mixed type iterables) is.
I agree with you. Every range that operates on a range of ranges must
support different types of ranges. After all, chain does that:
auto a = iota(10);
auto b = [3.1, 6.2, 9.3];
auto c = iota(11).map!q{a * a};
auto r = chain(a, b, c);
Ali
Re: Playing with ranges and ufcs
On Wednesday, February 27, 2013 23:48:05 bearophile wrote:
Andrea Fontana:
writeln(iota(10).cycle()[5..2].take(4));
print:
[5, 6, 7, 8]
Shouldn't [5..2] slice throw a compile/runtime error?
Please file it in bugzilla.
The opSlice of cycle() lacks those pre-conditions or tests, and
there are not enough unittests in Phobos to catch this simple bug:
auto opSlice(size_t i, size_t j)
{
auto retval = this.save;
retval._index += i;
return takeExactly(retval, j - i);
}
What it should be doing is using version(assert) to throw a RangeError if the
arguments are invalid, but the addition of version(assert) is quite recent
(previously, the best that could have been done was to assert). That's the
general policy, but it's not always followed like it should be.
- Jonathan M Davis
Re: Playing with ranges and ufcs
On Wednesday, 27 February 2013 at 22:48:06 UTC, bearophile wrote:
Andrea Fontana:
writeln(iota(10).cycle()[5..2].take(4));
print:
[5, 6, 7, 8]
Shouldn't [5..2] slice throw a compile/runtime error?
Please file it in bugzilla.
The opSlice of cycle() lacks those pre-conditions or tests, and
there are not enough unittests in Phobos to catch this simple
bug:
auto opSlice(size_t i, size_t j)
{
auto retval = this.save;
retval._index += i;
return takeExactly(retval, j - i);
}
j - i is positive because those numbers are unsigned, and
because D lacks a run-time errors for integral values because
the stupid Walter thinks those run-time are too much slow (it's
not actually true. Even Clang designers have understood it),
and Don thinks that Everything is Fine in D:
import std.stdio;
struct Foo {
auto opSlice(size_t i, size_t j) {
writeln(j - i); // Prints: 4294967293
}
}
void main() {
Foo f;
f[5 .. 2];
}
Maybe we need to run something similar to QuickCheck
(http://en.wikipedia.org/wiki/QuickCheck ) on Phobos.
Bye,
bearophile
Done!
Re: Playing with ranges and ufcs
Andrea Fontana: Done! I was about to file it myself, so I have added a bit more meat to your bug report. Bye, bearophile
Re: Playing with ranges and ufcs
Jonathan M Davis:
What it should be doing is using version(assert) to throw a
RangeError if the
arguments are invalid, but the addition of version(assert) is
quite recent
(previously, the best that could have been done was to assert).
Do you mean something like this?
auto opSlice(size_t i, size_t j) {
version(assert) {
if (i j)
throw new RangeError(some message here);
}
auto retval = this.save;
retval._index += i;
return takeExactly(retval, j - i);
}
What's the advantage of that compared to using a normal contract?
auto opSlice(size_t i, size_t j)
in {
assert(i = j, some message here);
} body {
auto retval = this.save;
retval._index += i;
return takeExactly(retval, j - i);
}
Bye,
bearophile
Re: Playing with ranges and ufcs
auto opSlice(size_t i, size_t j)
in {
assert(i = j, some message here);
} body {
auto retval = this.save;
retval._index += i;
return takeExactly(retval, j - i);
}
This is the bug opened by Andrea Fontana for this thread:
http://d.puremagic.com/issues/show_bug.cgi?id=9612
At its start I show this code, that contains a but that the D
compiler is able to catch at compile-time:
void main() {
auto a = [0,1,2,3,4,5,6,7,8,9][5 .. 2];
}
To do the same with user-defined structures time ago I have
suggested this, that is currently closed waiting for a better
solution:
http://d.puremagic.com/issues/show_bug.cgi?id=5906
Bye,
bearophile
Re: Playing with ranges and ufcs
To do the same with user-defined structures time ago I have suggested this, that is currently closed waiting for a better solution: What I meant to say is that if the assert(i = j) is inside the pre-condition then there's a hope to run it at compile time introducing some new trick inside the D language. But if you put code inside a version(assert) inside the body of opSlice() then the possibility of inventing a trick to do it is much lower. Bye, bearophile
Re: nWayUnion(tuple)?
On Wednesday, 27 February 2013 at 19:36:44 UTC, bearophile wrote: But in some cases I'd like to merge different types of ranges, that I can't put in an array. Is this use case worth supporting (usually the number of ranges is small so for such use cases a heap is not so needed)? I'm not sure how common the use case is, but I think it'd be fairly easy to support. Just internally have an array of indices to the tuple and use the heap with a less defined like myTup[a] myTup[b] to use the indices to look into the tuple to sort the indices appropriately. Just add some compile-time checks to make sure all of the ElementTypes of the tuple agree and it's essentially the same thing as already implemented. It actually probably wouldn't be a terrible idea to write a wrapper range that does this type of process so that it may be used with anything and the wrapper range could be a RandomAccessRange... this would (probably) make it possible to use tuples in a lot of places it isn't exactly allowed right now.
Re: Playing with ranges and ufcs
On Thursday, February 28, 2013 00:46:56 bearophile wrote: To do the same with user-defined structures time ago I have suggested this, that is currently closed waiting for a better solution: What I meant to say is that if the assert(i = j) is inside the pre-condition then there's a hope to run it at compile time introducing some new trick inside the D language. But if you put code inside a version(assert) inside the body of opSlice() then the possibility of inventing a trick to do it is much lower. Honestly, I think that that's a complete pipe dream anyway, but the point is to make it act like arrays, and they use RangeError. - Jonathan M Davis
Re: Playing with ranges and ufcs
On Thursday, February 28, 2013 00:35:08 bearophile wrote:
Jonathan M Davis:
What it should be doing is using version(assert) to throw a
RangeError if the
arguments are invalid, but the addition of version(assert) is
quite recent
(previously, the best that could have been done was to assert).
Do you mean something like this?
auto opSlice(size_t i, size_t j) {
version(assert) {
if (i j)
throw new RangeError(some message here);
}
auto retval = this.save;
retval._index += i;
return takeExactly(retval, j - i);
}
What's the advantage of that compared to using a normal contract?
Because then it more closely matches how arrays work. The only part that
doesn't is that it's fully tied to -release rather than -noboundschecked.
- Jonathan M Davis
Re: Playing with ranges and ufcs
Jonathan M Davis: Because then it more closely matches how arrays work. The only part that doesn't is that it's fully tied to -release rather than -noboundschecked. I see, thank you. Honestly, I think that that's a complete pipe dream anyway, I will keep dreaming for some more decades. In technology progress comes from dreamers that have the skills to create things like the Whiley (http://whiley.org/ ) language. Bye, bearophile
Re: nWayUnion(tuple)?
On Wednesday, 27 February 2013 at 23:54:31 UTC, Chris Cain wrote: I'm not sure how common the use case is, but I think it'd be fairly easy to support. Just internally have an array of indices to the tuple and use the heap with a less defined like myTup[a] myTup[b] to use the indices to look into the tuple to sort the indices appropriately. Just add some compile-time checks to make sure all of the ElementTypes of the tuple agree and it's essentially the same thing as already implemented. It actually probably wouldn't be a terrible idea to write a wrapper range that does this type of process so that it may be used with anything and the wrapper range could be a RandomAccessRange... this would (probably) make it possible to use tuples in a lot of places it isn't exactly allowed right now. I thought about this a bit more and I see that this is simply not a solution to the problem at all. I'm playing around with some things and if I come up with a solution in code, I'll post it.
Re: A little of coordination for Rosettacode
A possible idea is to translate this last Python version to D: http://rosettacode.org/wiki/Hamming_numbers#Cyclic_generator_method_.232. - - - - - - - - - - - - Another idea is to add a D version of Merge Sort that works with just a Input Range (like a single linked list, as a SList): http://rosettacode.org/wiki/Merge_sort Slicing the input range in two is probably possible. But what's the result of such function? Just an array created with an Appender? It can't be the same type of the input, because it is just a input range, so it doesn't need to have a put(). Maybe a given sink that is an output range? Or maybe a sink created inside given its type as template argument? Bye, bearophile
Re: nWayUnion(tuple)?
Chris Cain: I'm playing around with some things and if I come up with a solution in code, I'll post it. If you have some useful idea then please add it to the ER itself: http://d.puremagic.com/issues/show_bug.cgi?id=9611 Bye, bearophile
Re: nWayUnion(tuple)?
On Thursday, 28 February 2013 at 00:44:58 UTC, bearophile wrote: Chris Cain: I'm playing around with some things and if I come up with a solution in code, I'll post it. If you have some useful idea then please add it to the ER itself: http://d.puremagic.com/issues/show_bug.cgi?id=9611 Bye, bearophile I'll post this to the ER as well, but here you go :) https://gist.github.com/Zshazz/c4da4c3e0099062ab7e5
Re: nWayUnion(tuple)?
On Thursday, 28 February 2013 at 01:01:16 UTC, Chris Cain wrote: I'll post this to the ER as well, but here you go :) https://gist.github.com/Zshazz/c4da4c3e0099062ab7e5 And upon reading the ER, I see that this is (essentially) the solution it had. *doh* And I thought I was being pretty clever with this stuff :-p But in any case, either a wrapper that does this or the code in nWayUnion itself could easily do this.
Re: Make a hash out of two ranges
On 2/28/13, Andrej Mitrovic andrej.mitrov...@gmail.com wrote:
import std.algorithm;
import std.range;
void main()
{
auto r = map!((a, b) = [a : b])([foo], [bar]); // error
assert(r.front == [foo : bar]);
}
This doesn't compile, what am I missing?
A more clearer example:
void main()
{
auto r = map!((a, b) = [a : b])([1, 3], [2, 4]);
assert(r[0] == [1 : 2]);
assert(r[1] == [3 : 4]);
}
Re: Make a hash out of two ranges
On Thursday, February 28, 2013 04:56:48 Andrej Mitrovic wrote:
import std.algorithm;
import std.range;
void main()
{
auto r = map!((a, b) = [a : b])([foo], [bar]); // error
assert(r.front == [foo : bar]);
}
This doesn't compile, what am I missing?
Map only works on one range. You can't pass it two of them at the same time.
You'd need to wrap them in a single range of pairs with something like zip.
- Jonathan M Davis
Re: Make a hash out of two ranges
On Thursday, 28 February 2013 at 03:59:16 UTC, Andrej Mitrovic
wrote:
void main()
{
auto r = map!((a, b) = [a : b])([1, 3], [2, 4]);
assert(r[0] == [1 : 2]);
assert(r[1] == [3 : 4]);
}
Map in std.algorithm doesn't really work like that. Many
languages use map to mean hashing items, but map in
functional programming (and D) really means something more like
apply this function to each item.
So, if you did something like this:
auto arr = [1,3];
auto r = map!(a = a + 1)(arr);
assert(r[0] == 2); // because arr[0] + 1 == 2 ... f(arr[0]) =
f(1) = 1 + 1 = 2
assert(r[1] == 4); // because arr[1] + 1 == 4 ... f(arr[1]) =
f(3) = 3 + 1 = 4
Then you'd have the correct idea.
If you're trying to create an associative array from two arrays,
zip and associative arrays could be handy:
http://dlang.org/phobos/std_range.html#.zip
http://dlang.org/hash-map.html
So, you could use it like so...
int[int] aa;
foreach(a, b; zip([1,3], [2,4]))
aa[a] = b;
writeln(aa); // prints [1:2, 3:4]
Re: Transparent ANSI to UTF-8 conversion
On Wednesday, 27 February 2013 at 10:56:16 UTC, Lubos Pintes wrote: Hi, I would like to transparently convert from ANSI to UTF-8 when dealing with text files. For example here in Slovakia, virtually every text file is in Windows-1250. If someone opens a text file, he or she expects that it will work properly. So I suppose, that it is not feasible to tell someone if you want to use my program, please convert every text to UTF-8. A while back I wrote a little code that effectively does that, mind you it's probably not the right specific encoding, however you should be able to find the code points and replace them. I think this was for iso-8859-1. See Reading ASCII file with some codes above 127 (exten ascii) http://forum.dlang.org/thread/lehgyzmwewgvkdgra...@forum.dlang.org
Re: Make a hash out of two ranges
On 2/28/13, Jonathan M Davis jmdavisp...@gmx.com wrote:
You'd need to wrap them in a single range of pairs with something like zip.
I'm not exactly sure how to do that though. Just using zip in the
argument place won't work.
I've tried to create my own range, but I'm having issues:
import std.string;
import std.stdio;
import std.range;
auto mymap(alias func, Ranges...)(Ranges ranges)
{
static struct MyMap
{
Ranges ranges;
@property bool empty()
{
foreach (ref range; ranges)
if (range.empty)
return true;
return false;
}
@property auto front()
{
static string getMixin()
{
string[] args;
foreach (idx; 0 .. Ranges.length)
args ~= format(ranges[%s].front, idx);
return args.join(, );
}
mixin(format(q{
return func(%s);
}, getMixin()));
}
void popFront()
{
foreach (ref range; ranges)
range.popFront();
}
}
return MyMap(ranges);
}
void main()
{
auto r = mymap!((a, b) = [a : b])([1, 3], [2, 4]);
writeln(r.front); // [1:2]
r.popFront();
writeln(r.front); // [3:4]
r.popFront();
// writeln(r.front); // this would throw as expected
auto r2 = mymap!((a, b) = [a : b])([1, 3], [2, 4]);
foreach (x; r2)
writeln(r2); // prints [[1:2], [3:4]] twice, why?
}
Re: Make a hash out of two ranges
On 2/28/13, Andrej Mitrovic andrej.mitrov...@gmail.com wrote: foreach (x; r2) writeln(r2); // prints [[1:2], [3:4]] twice, why? Oh dumb me I'm priting r2 instead of x, my bad.
Re: Make a hash out of two ranges
On 2/28/13, Chris Cain clc...@uncg.edu wrote: Map in std.algorithm doesn't really work like that. Many languages use map to mean hashing items, but map in functional programming (and D) really means something more like apply this function to each item. I know that, I was expecting it to work with multiple ranges if the functor takes multiple arguments. If you're trying to create an associative array from two arrays A *lazy* range of associative arrays. On 2/28/13, bearophile bearophileh...@lycos.com wrote: See: http://dlang.org/phobos/std_array.html#.assocArray Same answer as above. My custom range implementation shows exactly what I was trying to use map for. It only seems natural to me to have this ability, but maybe map!() can't be customized for this use. Anyway I'm satisfied with my approach unless you can make map!() do the same thing with some zip tricks or something.
Re: Make a hash out of two ranges
On Thursday, 28 February 2013 at 04:50:34 UTC, Andrej Mitrovic wrote: On 2/28/13, Chris Cain clc...@uncg.edu wrote: Map in std.algorithm doesn't really work like that. Many languages use map to mean hashing items, but map in functional programming (and D) really means something more like apply this function to each item. I know that, I was expecting it to work with multiple ranges if the functor takes multiple arguments. If you're trying to create an associative array from two arrays A *lazy* range of associative arrays. Aha, I see now. Here you go: zip([1,3], [2,4]) .map!(e = [e[0] : e[1]])() .writeln(); zip gives out a tuple, so I wish it were possible to do .map!((a,b) = [a : b])() instead and have it appropriately use the elements of the tuple. Maybe that'd be a potential enhancement possibility for map?
