Re: sort, .array and folding on immutable data (finding most common character in column of matrix)
On Monday, 19 December 2016 at 14:09:47 UTC, pineapple wrote: This is a shortcoming of Phobos - here is a package of sorting algorithms including some that do not require their inputs to be mutable, random access, and/or finite: https://github.com/pineapplemachine/mach.d/tree/master/mach/range/sort Oh nice! Will take a looksies It's worth noting that giving up eagerness, random access, etc. often comes with a speed penalty. It may be more efficient just to copy the lazy things into memory first and then sort in-place, as you have been doing. True dat!
Re: sort, .array and folding on immutable data (finding most common character in column of matrix)
On Monday, 19 December 2016 at 12:45:48 UTC, Nicholas Wilson wrote: Ah, oh well. It was nice in theory. Indeed. Thank you for trying Nicholas :) auto word = data.map!(reduce!max).array.map!"a[1]".array; you want auto word = data.map!"a[1]".map!(reduce!max).array; Problem max has to performed on the frequency part of the tuple and "word" has to result in a concatenation of all the character parts of the highest frequencied letters. So that up there will result in "word" = 364782 // or something.
Re: Returning structs from COM
On Monday, 19 December 2016 at 22:47:26 UTC, kinke wrote: On Monday, 19 December 2016 at 09:49:13 UTC, kinke wrote: [...] I did some research myself and indeed, COM classes/interfaces are apparently subject to a separate ABI. Unfortunately, googling it hasn't turned up any official (and not even some inofficial) documentation so far. Based on the first few tests on Win64, integers are returned in RAX, floats (and I guess doubles too) in XMM0, and structs (incl. 2x int32 and 2x float) via hidden sret pointer, with `this` pointer in RCX (1st arg) and `sret` in RDX (2nd arg). Compared to the normal Win64 C++ ABI it just seems more conservative by always returning structs via hidden pointer. Wow! That was quick. Such tests would take me few days. I'm not sure how it may help but there is "specs" in MS .net core runtime, just search the repo for COM like this https://github.com/dotnet/coreclr/blob/32f0f9721afb584b4a14d69135bea7ddc129f755/src/vm/amd64/GenericComCallStubs.asm#L177
Re: Returning structs from COM
https://issues.dlang.org/show_bug.cgi?id=16987 https://github.com/ldc-developers/ldc/issues/1935
Re: Returning structs from COM
On Monday, 19 December 2016 at 09:49:13 UTC, kinke wrote: That's rather interesting. The COM function really seems to return the struct directly, not returning an HRESULT and setting some output pointee as most COM functions I've seen so far. According to the Win64 ABI, the returned D2D1_SIZE_F struct should be returned in RAX, as the 2 floats are <= 64 bit. But your workaround seems to suggest it's using sret (struct-return via hidden pointer). To make sure this is the case, I'd suggest inspecting the C++ assembly for a trivial function. If so, we'll need to find out why the Win64 ABI isn't followed and whether COM has its own ABI. I did some research myself and indeed, COM classes/interfaces are apparently subject to a separate ABI. Unfortunately, googling it hasn't turned up any official (and not even some inofficial) documentation so far. Based on the first few tests on Win64, integers are returned in RAX, floats (and I guess doubles too) in XMM0, and structs (incl. 2x int32 and 2x float) via hidden sret pointer, with `this` pointer in RCX (1st arg) and `sret` in RDX (2nd arg). Compared to the normal Win64 C++ ABI it just seems more conservative by always returning structs via hidden pointer.
Re: std.experimental.logger + threadIds
On 19/12/2016 21:32, Robert burner Schadek wrote:
> The ugly way is to create a @trusted function/lambda that coverts the
> threadId to a string.
>
> Not sure about the pretty way.
thanks a lot. works good enough, just for reference, I added:
string tid2string(Tid id) @trusted {
import std.conv : text;
return text(id);
}
Re: std.experimental.logger + threadIds
The ugly way is to create a @trusted function/lambda that coverts the threadId to a string. Not sure about the pretty way.
Re: Swap front for char[] input ranges
On 12/19/2016 06:09 AM, RazvanN wrote:
> On Monday, 19 December 2016 at 12:25:02 UTC, Ali Çehreli wrote:
>> On 12/19/2016 02:41 AM, RazvanN wrote:
>> > [...]
>>
>> As your comments make it clear below, they cannot be InputRanges.
>>
>> > [...]
>> swapping code
>> > [...]
>>
>> Obivously, tmp1 and tmp2 are unusued there. :)
>>
>> > [...]
>> passed.
>> > [...]
>> state
>> > [...]
>> is:
>> > [...]
>> input ranges
>> > [...]
>>
>> Not possible... It's ok to use something similar to the following
>> template constraint:
>>
>> void foo(R1, R2)(R1 r1, R2 r2)
>> if ((hasSwappableElements!R1 && hasSwappableElements!R2) ||
>> (hasLvalueElements!R1 && hasLvalueElements!R2)) {
>> // ...
>> }
>>
>> Ali
>
> In this case, the bringToFront function [1] should not accept char[]
> as parameters? Or a special path should be added in the function, so
> that char[] are treated specially?
>
> [1] http://dlang.org/phobos/std_algorithm_mutation.html#bringToFront
First, thanks to you and to your colleagues very much for improving D
with these fixes. :)
I think you're working on the following bug:
https://issues.dlang.org/show_bug.cgi?id=16959
Regarding the compilation error inside swapFront, its template
constraints should be fixed as I suggested earlier. It cannot work with
any two InputRanges. I think it warrants its separate bug.
private void swapFront(R1, R2)(R1 r1, R2 r2)
if (isInputRange!R1 && isInputRange!R2)
{
static if (is(typeof(swap(r1.front, r2.front
{
swap(r1.front, r2.front);
}
else
{
auto t1 = moveFront(r1), t2 = moveFront(r2);
r1.front = move(t2);
r2.front = move(t1);
}
}
Regarding bringToFront:
1) Reading its documentation, bringToFront() seems to be an algorithms
that mutates its ranges (not just give a brought-to-front view of
existing data). If so, its argument cannot simply be an InputRange
because there are InputRanges out there where you cannot write it their
.front. (char[] is just one example.):
size_t bringToFront(Range1, Range2)(Range1 front, Range2 back)
if (isInputRange!Range1 && isForwardRange!Range2)
{
// ...
}
Its template constraint must also be changed to include a combination of
hasLvalueElements() and hasMobileElements().
2) After fixing that, a char[] can indeed bring characters to front but
it would be an expensive operation where a multi-byte Unicode character
would necessarily move a single-byte Unicode character to the right.
(Additionally, depending on its length, the first argument may allow
only a partial UTF-8 encoding at its end. Fail! :) )
I don't know how to allow or encode such an expensive operation which is
outside the documented "Ο(max(front.length, back.length))" complexity of
bringToFront(). If it were me, I would look for possibilities to change
the behavior and make bringToFront() a non-mutating algorithm.
What do others think?
Ali
std.experimental.logger + threadIds
I am experimenting with the logger interface and want to write a custom logger, that also outputs the threadID or Tid of the LogEntries. The documentation shows how to do a custom logger, but I am unable to convert the threadId to a string, because all conversion functions are not @safe. Is there a way around this? Thanks in advance, Christian
Re: Pointer to private structure
On Monday, 19 December 2016 at 10:14:49 UTC, Ali wrote: On Monday, 19 December 2016 at 06:42:27 UTC, Nikhil Jacob wrote: [...] What're you trying to do here? Forward declarations in C++ are used to solve a few different things: 1. Reduce build times (unneeded in D AFAIK) 2. Break cyclic references (unneeded in D again?) 3. Give APIs visibility (D's modules and Access layers solve this) 4. Maintain binary compatibility while allowing internal data changes (aka pimlp idiom) <-- This I believe you cannot do in D - https://wiki.dlang.org/Access_specifiers_and_visibility (someone correct me if I'm wrong) I've seen something about .di files in D. But they seem flakey a bit. I was trying to do something similar to pimlp idiom. But after thinking over it, I found a better way in D. Thanks for pointing to the wiki
Re: Is it possbile to specify a remote git repo as dub dependency?
On Monday, 19 December 2016 at 14:45:07 UTC, biocyberman wrote: On Monday, 19 December 2016 at 14:18:17 UTC, Jacob Carlborg wrote: On 2016-12-19 13:11, biocyberman wrote: I can write a short script to clone the remote git repo and use it as a submodule. But if it is possible to do with dub, it will be more convenient. It's not currently possible. I see, it is both a good thing and a bad thing. Good thing is to encourage developers to submit packages to central dub registry. Bad thing is, when that does not happen soon enough, other developers who use the package will have to do something for themselves. To be honest there is no need for dub registry at all. Vim plugin managers can pull plugins from every github repository. Javascript ended up with 10 dependency managers every of which has its own registry (and npm as the most official of them). And the js case will happen to D aswell if it become more popular.
Re: Is it possbile to specify a remote git repo as dub dependency?
On Monday, 19 December 2016 at 14:45:07 UTC, biocyberman wrote: I see, it is both a good thing and a bad thing. Good thing is to encourage developers to submit packages to central dub registry. Bad thing is, when that does not happen soon enough, other developers who use the package will have to do something for themselves. What you can do for private package as of today is: - use path-based dependencies and put your packages in the same repo - use git submodules and path-based dub dependencies together If it's a public package, you can register yourself on the DUB repositery.
Re: Is it possbile to specify a remote git repo as dub dependency?
On Monday, 19 December 2016 at 14:18:17 UTC, Jacob Carlborg wrote: On 2016-12-19 13:11, biocyberman wrote: I can write a short script to clone the remote git repo and use it as a submodule. But if it is possible to do with dub, it will be more convenient. It's not currently possible. I see, it is both a good thing and a bad thing. Good thing is to encourage developers to submit packages to central dub registry. Bad thing is, when that does not happen soon enough, other developers who use the package will have to do something for themselves.
Re: Is it possbile to specify a remote git repo as dub dependency?
On 2016-12-19 13:11, biocyberman wrote: I can write a short script to clone the remote git repo and use it as a submodule. But if it is possible to do with dub, it will be more convenient. It's not currently possible. -- /Jacob Carlborg
Re: sort, .array and folding on immutable data (finding most common character in column of matrix)
On Monday, 19 December 2016 at 09:24:38 UTC, Ali wrote: Ok so laziness stops as soon as sort is required on a range then? Ahh, because in place algorithms? Are there any plans in D to make is to that you can output copies to collections so that you could do something like filter.transpose.sort and just have it output a modified copy? This is a shortcoming of Phobos - here is a package of sorting algorithms including some that do not require their inputs to be mutable, random access, and/or finite: https://github.com/pineapplemachine/mach.d/tree/master/mach/range/sort The library is permissively licensed; feel free to take out whatever you need. It's worth noting that giving up eagerness, random access, etc. often comes with a speed penalty. It may be more efficient just to copy the lazy things into memory first and then sort in-place, as you have been doing.
Re: Swap front for char[] input ranges
On Monday, 19 December 2016 at 12:25:02 UTC, Ali Çehreli wrote:
On 12/19/2016 02:41 AM, RazvanN wrote:
> [...]
As your comments make it clear below, they cannot be
InputRanges.
> [...]
swapping code
> [...]
Obivously, tmp1 and tmp2 are unusued there. :)
> [...]
passed.
> [...]
state
> [...]
is:
> [...]
input ranges
> [...]
Not possible... It's ok to use something similar to the
following template constraint:
void foo(R1, R2)(R1 r1, R2 r2)
if ((hasSwappableElements!R1 && hasSwappableElements!R2) ||
(hasLvalueElements!R1 && hasLvalueElements!R2)) {
// ...
}
Ali
In this case, the bringToFront function [1] should not accept
char[]
as parameters? Or a special path should be added in the function,
so that char[] are treated specially?
[1]
http://dlang.org/phobos/std_algorithm_mutation.html#bringToFront
Re: Returning structs from COM
On Monday, 19 December 2016 at 12:23:46 UTC, evilrat wrote: x64 still has messed up 'this'(both dmd and ldc, not tested with gdc) Please file an LDC issue then, as I'm pretty sure it works for regular (non-COM) C++ classes.
Re: sort, .array and folding on immutable data (finding most common character in column of matrix)
On Monday, 19 December 2016 at 11:42:55 UTC, Ali wrote: On Monday, 19 December 2016 at 10:03:34 UTC, Nicholas Wilson wrote: [...] The seedless version without the typeof(a)(b[0], b[1]) hack (with default inited T) seems to crap out with: [...] Ah, oh well. It was nice in theory. [...] auto word = data.map!(reduce!max).array.map!"a[1]".array; you want auto word = data.map!"a[1]".map!(reduce!max).array; which will try to take the max of a single variable, not a tuple, which should succeed.
Re: Returning structs from COM
On Monday, 19 December 2016 at 09:49:13 UTC, kinke wrote:
On Saturday, 3 December 2016 at 09:51:00 UTC, John C wrote:
Some DirectX methods return structs by value, but when I try
calling them I either get garbage or an access violation.
Usually COM methods return structs by pointer as a parameter,
but these are returning the struct as the actual return value,
as in this definition:
extern(Windows):
struct D2D1_SIZE_F { float width, height; }
interface ID2D1Bitmap : ID2D1Image {
D2D1_SIZE_F GetSize();
}
If I rewrite GetSize to return by pointer as a parameter, it
appears to work and I get the correct width and height without
an AV being thrown. And I can add a helper method that returns
by value:
interface ID2D1Bitmap : ID2D1Image {
void GetSize(D2D1_SIZE_F* size);
final D2D1_SIZE_F GetSize() {
D2D1_SIZE_F size;
GetSize();
return size;
}
}
yes it works indeed O_o
also align(8) for struct seems to work in x86 but has some issues
with ESP(reserved 4 bytes for pointer, still has 4 bytes for
second float?), crashes without align at later point.
x64 still has messed up 'this'(both dmd and ldc, not tested with
gdc),
but with passing parameter instead of return it really keeps
stack where it leaves it and even 'this' are in place so
everything is seems to work.
Re: Swap front for char[] input ranges
On Monday, 19 December 2016 at 10:41:46 UTC, RazvanN wrote:
Hi,
I have a function which accepts 2 input Ranges and swaps the
first element in Range1 with the first element in Range2. The
swapping code looks something like this :
static if (is(typeof(swap(r1.front, r2.front
{
swap(r1.front, r2.front);
}
else
{
auto t1 = moveFront(r1), t2 = moveFront(r2);
auto tmp1 = r1.front;
auto tmp2 = r2.front;
r1.front = move(t2);
r2.front = move(t1);
}
This code works for most cases, except when 2 char[] are passed.
In that case, the compilation fails with error messages which
state
that r1.front and r2.front are not Lvalues. So, my question is:
how can I swap the 2 elements since in the case of char[] input
ranges
the front property does not return a reference?
does it work if you cast the 2 char[] as 2 ubyte[] ?
If so, it's a problem with narrow strings and utf8 decoding...i.e
there's no guarantee that each code unit in r1 matches to a code
unit in r2.
Is it possbile to specify a remote git repo as dub dependency?
I can write a short script to clone the remote git repo and use it as a submodule. But if it is possible to do with dub, it will be more convenient.
Re: sort, .array and folding on immutable data (finding most common character in column of matrix)
On Monday, 19 December 2016 at 10:03:34 UTC, Nicholas Wilson
wrote:
On Monday, 19 December 2016 at 09:24:38 UTC, Ali wrote:
On Monday, 19 December 2016 at 00:11:49 UTC, Nicholas Wilson
wrote:
[...]
Ok so laziness stops as soon as sort is required on a range
then?
No. Because a lazy range is not random access, and therefore
does not meet sorts requirement.
Ahh, because in place algorithms?
Yes
Are there any plans in
D to make is to that you can output copies to collections so
that you could do something like filter.transpose.sort and
just have it output a modified copy?
None that I know.
[...]
Hmm, for the other problem you could do
chain(only(T(dchar.max, uint.max)),range)
and still use the seedless version.
Thanks for input so far!
The seedless version without the typeof(a)(b[0], b[1]) hack (with
default inited T) seems to crap out with:
"Unable to deduce an acceptable seed type for __lambda2 with
element type immutable(Tuple!(dchar, uint))."
BTW: Your chain fix does indeed make the T(dchar.init, uint.init)
seeded fold work on a seedless version of fold :D. Only problem
is the need for the typeof hack again though.
Some more info: The following alternate version of this program
works great.
// Swapping the tuple elements that are produced by "group"
around is necessary for reduce!max to work.
auto data =
import("data_06.txt").split("\n").transposed.map!`a.array.sort().group.map!"tuple(a[1], a[0])".array`.array;
void main() {
auto word = data.map!(reduce!max).array.map!"a[1]".array;
word.writeln;
}
But this stops working as soon as you take the "word" variable
outside the scope of main and in to global scope. If you make
everything static immutable then you get:
"Error: static assert "Unable to deduce an acceptable seed type
for std.algorithm.comparison.max with element type
immutable(Tuple!(uint, dchar))."
Swap front for char[] input ranges
Hi,
I have a function which accepts 2 input Ranges and swaps the
first element in Range1 with the first element in Range2. The
swapping code looks something like this :
static if (is(typeof(swap(r1.front, r2.front
{
swap(r1.front, r2.front);
}
else
{
auto t1 = moveFront(r1), t2 = moveFront(r2);
auto tmp1 = r1.front;
auto tmp2 = r2.front;
r1.front = move(t2);
r2.front = move(t1);
}
This code works for most cases, except when 2 char[] are passed.
In that case, the compilation fails with error messages which
state
that r1.front and r2.front are not Lvalues. So, my question is:
how can I swap the 2 elements since in the case of char[] input
ranges
the front property does not return a reference?
Re: Pointer to private structure
On Monday, 19 December 2016 at 06:42:27 UTC, Nikhil Jacob wrote: On Monday, 19 December 2016 at 06:21:10 UTC, ketmar wrote: i bet that just trying this with D compiler will take less time than writing forum post. I did try but it seems to give compilation failure... Let me try once more and I will get back with more details. What're you trying to do here? Forward declarations in C++ are used to solve a few different things: 1. Reduce build times (unneeded in D AFAIK) 2. Break cyclic references (unneeded in D again?) 3. Give APIs visibility (D's modules and Access layers solve this) 4. Maintain binary compatibility while allowing internal data changes (aka pimlp idiom) <-- This I believe you cannot do in D - https://wiki.dlang.org/Access_specifiers_and_visibility (someone correct me if I'm wrong) I've seen something about .di files in D. But they seem flakey a bit.
Re: sort, .array and folding on immutable data (finding most common character in column of matrix)
On Monday, 19 December 2016 at 09:24:38 UTC, Ali wrote: On Monday, 19 December 2016 at 00:11:49 UTC, Nicholas Wilson wrote: [...] Ok so laziness stops as soon as sort is required on a range then? No. Because a lazy range is not random access, and therefore does not meet sorts requirement. Ahh, because in place algorithms? Yes Are there any plans in D to make is to that you can output copies to collections so that you could do something like filter.transpose.sort and just have it output a modified copy? None that I know. [...] Hmm, for the other problem you could do chain(only(T(dchar.max, uint.max)),range) and still use the seedless version. Thanks for input so far!
Re: Returning structs from COM
On Saturday, 3 December 2016 at 09:51:00 UTC, John C wrote:
Some DirectX methods return structs by value, but when I try
calling them I either get garbage or an access violation.
Usually COM methods return structs by pointer as a parameter,
but these are returning the struct as the actual return value,
as in this definition:
extern(Windows):
struct D2D1_SIZE_F { float width, height; }
interface ID2D1Bitmap : ID2D1Image {
D2D1_SIZE_F GetSize();
}
If I rewrite GetSize to return by pointer as a parameter, it
appears to work and I get the correct width and height without
an AV being thrown. And I can add a helper method that returns
by value:
interface ID2D1Bitmap : ID2D1Image {
void GetSize(D2D1_SIZE_F* size);
final D2D1_SIZE_F GetSize() {
D2D1_SIZE_F size;
GetSize();
return size;
}
}
But does anyone know why the original definition works in C++
but not D? Is it a bug? (I'm compiling with -m64.)
That's rather interesting. The COM function really seems to
return the struct directly, not returning an HRESULT and setting
some output pointee as most COM functions I've seen so far.
According to the Win64 ABI, the returned D2D1_SIZE_F struct
should be returned in RAX, as the 2 floats are <= 64 bit. But
your workaround seems to suggest it's using sret (struct-return
via hidden pointer). To make sure this is the case, I'd suggest
inspecting the C++ assembly for a trivial function. If so, we'll
need to find out why the Win64 ABI isn't followed and whether COM
has its own ABI.
I know for ldc that function that return struct by value
actually return by a hidden return parameter pointer.
Not always. In fact, it highly depends on the target ABI which
structs are returned in registers and which ones via sret.
https://issues.dlang.org/show_bug.cgi?id=16527
That is definitely a bug in DMD (swapping `this` and `sret`
pointers) but doesn't apply to LDC, and is a separate issue.
Re: sort, .array and folding on immutable data (finding most common character in column of matrix)
On Monday, 19 December 2016 at 00:11:49 UTC, Nicholas Wilson wrote: On Sunday, 18 December 2016 at 22:26:50 UTC, Ali wrote: 1. The first line with the splitting, I need to use .array three times. The last one I understand is because on "line 2" I alias T as the type of the data, and if I don't call .array then it's a MapResult type which has no opIndex. Yes? 2. On "line 1" I have to call .array.sort(). Why can't I just call .sort() on the transposed rows? Because sort requires a random access range. Ok so laziness stops as soon as sort is required on a range then? Ahh, because in place algorithms? Are there any plans in D to make is to that you can output copies to collections so that you could do something like filter.transpose.sort and just have it output a modified copy? Unfortunately arrays have a builtin property sort that for some reason takes precedence of std.algorithm.sort when called without (). The compiler error is probably because .sort is a mutating operation and you're working with immutable data. Ah ok. I think it's just that array.sort doesn't have CTFE capabilities then: "Error: _adSort cannot be interpreted at compile time, because it has no available source code" Try })(ImmutableOf!T.init); or use the seedless version of fold. may need to import std.traits(?) Same error message :( does `data.map!fold!"a[1] > b[1] ? a : b".map!"a[0]".array;` work? Actually I guess what's stopping this is my problem above maybe? Since I can't jut return b because of that error. I can't use the seedless version of fold either because Another version of the problem requires the seed be T.init(dchar.max, uint.max). Thanks for input so far!
