Re: Final templated interface method not found

2015-09-15 Thread anonymous via Digitalmars-d-learn
On Wednesday 16 September 2015 06:14, Andre wrote:

> Hi,
> 
> following coding shoud work, or?
> It doesn't compile with v2.068.0.
> 
> Kind regards
> André
> 
> interface IfStatement
> {
>   void execute();
>   
>   final void execute(T...)(T t)
>   {
>   execute();
>   }
> }
> 
> class Statement: IfStatement
> {
>   void execute(){}
> }
> 
> void main()
> {
>   new Statement().execute(1,"Hello World");
> }

You need to add IfStatement.execute to the overload set like so:

class Statement: IfStatement
{
alias execute = IfStatement.execute;
void execute(){}
}



Re: Initalizing complex array types or some other problem ;/

2015-09-15 Thread anonymous via Digitalmars-d-learn
On Wednesday 16 September 2015 03:46, Prudence wrote:

> In any case, Maybe you are not as smart as you think you are if 
> you can't understand it?

kthxbye


Re: Load Qt UI XML File as GUI

2015-09-15 Thread Mike McKee via Digitalmars-d-learn
On Wednesday, 16 September 2015 at 04:36:15 UTC, Steven 
Schveighoffer wrote:

Seems to have moved here, but it doesn't look fresh:

https://bitbucket.org/qtd/


Yep, but when I poke around in the source, I can't see anywhere 
that the QtD can read the .ui files that QtCreator creates.





Re: Load Qt UI XML File as GUI

2015-09-15 Thread Steven Schveighoffer via Digitalmars-d-learn

On 9/16/15 12:03 AM, Mike McKee wrote:

Unfortunately, the http://dsource.org/forums/ doesn't appear to be
active -- I can't login after I registered. This is where the QtD
project has their forum. So, I'm asking this here.


Seems to have moved here, but it doesn't look fresh:

https://bitbucket.org/qtd/

dsource is dead (only there for archive reasons), don't use it.

-Steve


Re: chaining chain Result and underlying object of chain

2015-09-15 Thread Steven Schveighoffer via Digitalmars-d-learn

On 9/14/15 11:30 AM, Ali Çehreli wrote:

On 09/14/2015 08:01 AM, Laeeth Isharc wrote:
 > I was trying to use the same variable eg
 >
 >auto chain1 = chain("foo", "bar");
 >chain1 = chain(chain1, "baz");
[...]
 > It may be that the type of chain1
 > and chain2 don't mix.

Exactly.

I was going to recommend using pragma(msg, typeof(chain1)) to see what
they are but it looks like chain()'s return type is not templatized. (?)

 pragma(msg, typeof(chain1));
 pragma(msg, typeof(chain2));

Prints

Result
Result

instead of something like (hypothetical)

ChainResult!(string, string)
ChainResult!(ChainResult!(string, string), string)

Ali


typeid is a bit better:

import std.range;

void main()
{
import std.stdio;
auto chain1 = chain("hi", "there");
auto chain2 = chain(chain1, "friend");
writeln(typeid(chain1));
writeln(typeid(chain2));
}

output:

std.range.chain!(string, string).chain.Result
std.range.chain!(Result, string).chain.Result

I still see that "Result" as a parameter for chain2.

I think the compiler should be better at printing these types at compile 
time.


-Steve


Best Direction on Spawning Process Async

2015-09-15 Thread Mike McKee via Digitalmars-d-learn

What's the best direction from...

http://dlang.org/phobos/std_process.html

...on spawning an async process and then peeking at it 
occasionally as it runs, and then get notified when it finishes? 
In other words, what std.process functions would you recommend I 
use? What I want to avoid is a blocking state where the GUI 
freezes because it's waiting for the process to complete.


For instance, imagine you are building a front end GUI (like 
GtkD) to a command-line based antivirus scanner. You'll want to 
spawn the process, show a progress bar, and as the command line 
returns status information, you peek at it asynchronously and 
then update the progress bar (or perhaps store virus detected 
info in a table), and then stop the progress bar at 100% when the 
command process has finished.




Final templated interface method not found

2015-09-15 Thread Andre via Digitalmars-d-learn

Hi,

following coding shoud work, or?
It doesn't compile with v2.068.0.

Kind regards
André

interface IfStatement
{
void execute();

final void execute(T...)(T t)
{
execute();
}
}

class Statement: IfStatement
{
void execute(){}
}

void main()
{
new Statement().execute(1,"Hello World");
}


Re: Another, is it a bug?

2015-09-15 Thread Random D user via Digitalmars-d-learn
On Wednesday, 16 September 2015 at 03:54:34 UTC, Adam D. Ruppe 
wrote:

On Wednesday, 16 September 2015 at 03:48:59 UTC, Random D user
Given that, normally properties are just overloaded methods in 
D, it's pretty sad classes break this behavior/convention.


The D behavior for overloading is different in general:

http://dlang.org/hijack.html

It basically never overloads across scopes. You need to alias 
the name into the scope too explicitly


Thanks. That pretty much answers directly to all my questions. I 
tried to look for this info in class docs/reference, but couldn't 
find it (obviously). I never thought that this would be in 
"articles".


Load Qt UI XML File as GUI

2015-09-15 Thread Mike McKee via Digitalmars-d-learn
Unfortunately, the http://dsource.org/forums/ doesn't appear to 
be active -- I can't login after I registered. This is where the 
QtD project has their forum. So, I'm asking this here.


Is it possible with D and QtD to draw my GUI using QtCreator, and 
then take its UI XML file and load it somehow via QtD and D? That 
way, I don't need to do everything by hand and can utilize the 
power of a WYSIWYG form tool (like Glade), and then just 
manipulate the rest by code after that?





Re: Another, is it a bug?

2015-09-15 Thread Adam D. Ruppe via Digitalmars-d-learn

On Wednesday, 16 September 2015 at 03:48:59 UTC, Random D user
Given that, normally properties are just overloaded methods in 
D, it's pretty sad classes break this behavior/convention.


The D behavior for overloading is different in general:

http://dlang.org/hijack.html

It basically never overloads across scopes. You need to alias the 
name into the scope too explicitly


Re: Another, is it a bug?

2015-09-15 Thread Random D user via Digitalmars-d-learn

On Wednesday, 16 September 2015 at 03:17:05 UTC, Meta wrote:
Considering Father defines the function `int eat()` and 
Daughter defines the completely different function `int 
eat(int)`, it doesn't surprise me. You're not using virtual 
dispatch when you do `return super.eat` or `d.Father.eat()`, 
you're delegating the method call to the base class.


Yeah... I guess I was expecting it to overload across class 
boundaries. I mean there's already a member eat in base class and 
sub class can't override that since it's got different 
parameters, and it's a function (can't be variable), so the 
reasonable thing would be to overload it (which is why I tried 
override to see if it forces/hints overriding/overloading).
Instead it creates two ambiguous names of which only one has to 
be disambiguated to use which seems super error prone. IMO it 
should just be error/warning.


Given that, normally properties are just overloaded methods in D, 
it's pretty sad classes break this behavior/convention.


Re: Another, is it a bug?

2015-09-15 Thread Meta via Digitalmars-d-learn
On Wednesday, 16 September 2015 at 02:59:06 UTC, Random D user 
wrote:
I'm trying to make a base class with get property and a sub 
class with corresponding set property. The value for the base 
class is set via constructor.
The intuitive way doesn't seem to work and workarounds are 
unnecessarily ugly (considering you'll sprinkle them all over 
the codebase).


class Father
{
int eat()
{
return 1;
}
}

class Daughter : Father
{

void eat( int apples ) {}

// int eat() { return super.eat(); }// Workaround A, 
works as expected
//override int eat( int apples ) {} // Workaround D, 
fails -> Error: function main.Daughter.eat does not override 
any function, did you mean to override 'main.Father.eat'?

}

Daughter d = new Daughter();

// BUG? I expected this to work. It seems that compiler doesn't 
even look into parent class to see if there's a matching 
function.
//int num = d.eat();// Error: function 
main.Daughter.eat (int apples) is not callable using argument 
types ()


int num2 = (cast(Father)d).eat();   // Workaround B, works as 
expected
int num3 = d.Father.eat();  // Workaround C, works as 
well


Considering Father defines the function `int eat()` and Daughter 
defines the completely different function `int eat(int)`, it 
doesn't surprise me. You're not using virtual dispatch when you 
do `return super.eat` or `d.Father.eat()`, you're delegating the 
method call to the base class.


Another, is it a bug?

2015-09-15 Thread Random D user via Digitalmars-d-learn
I'm trying to make a base class with get property and a sub class 
with corresponding set property. The value for the base class is 
set via constructor.
The intuitive way doesn't seem to work and workarounds are 
unnecessarily ugly (considering you'll sprinkle them all over the 
codebase).


class Father
{
int eat()
{
return 1;
}
}

class Daughter : Father
{

void eat( int apples ) {}

// int eat() { return super.eat(); }// Workaround A, 
works as expected
//override int eat( int apples ) {} // Workaround D, 
fails -> Error: function main.Daughter.eat does not override any 
function, did you mean to override 'main.Father.eat'?

}

Daughter d = new Daughter();

// BUG? I expected this to work. It seems that compiler doesn't 
even look into parent class to see if there's a matching function.
//int num = d.eat();// Error: function 
main.Daughter.eat (int apples) is not callable using argument 
types ()


int num2 = (cast(Father)d).eat();   // Workaround B, works as 
expected

int num3 = d.Father.eat();  // Workaround C, works as well


Re: Initalizing complex array types or some other problem ;/

2015-09-15 Thread Prudence via Digitalmars-d-learn

On Tuesday, 15 September 2015 at 20:54:49 UTC, anonymous wrote:

On Tuesday 15 September 2015 22:09, Prudence wrote:


The code below doesn't work.


Please be more specific in how it doesn't work. Mention the 
error message if there is one, or say how the code behaves 
differently from what you'd expect.


Trying to compile the code (after kicking getch out), I get 
this error:

core.exception.RangeError@test.d(103): Range violation

Line 103 is: writeln(MyStore.Store[k].length);

I can't find where you set Store[k]. Maybe you forgot that or 
deleted it accidentally?


I made a guess and added this line in SingleStore.New:
o.Store[k] ~= tuple(v, o);

It still throws a range error with this. That's because 
associative arrays are a little weird, unfortunately.


An AA is effectively initialized on the first assignment. So 
(with my addition) the first SingleStore.New call initializes 
the AA. But it only initializes o.Store, not the original 
variable s (i.e. ObjectStore.Store). s is left empty.


Possible solutions/workarounds:
* Append to s[k], then assign s to o.Store.
* Initialize ObjectStore.Store in a static constructor:

static this()
{
Store[TKey.init] = [];
Store.remove(TKey.init);
}


Maybe. Seems to work without it. The code below should compile on 
your system and work(and prints 1 2 3 4, 4 3 2 1). The getch is 
required on windows if I want to see the output, so I don't know 
why you even bothered mention replacing it.






import std.stdio;
import std.concurrency;



extern (C) int getch();
import std.string;
import std.concurrency;
import core.time;
import core.thread;
import std.container.array;
import std.typecons;







public class SingleStore(TKey, TValue)
{
public TValue Value;
public TKey Key;
public Tuple!(TValue, SingleStore!(TKey, TValue))[][TKey] Store;


	// Duplicate entries will be removed together as there is no way 
to distinguish them

public auto Remove()
{
import std.algorithm;
if (Value == null || Key == null) return;
int count = 0;
for(int i = 0; i < Store[Key].length; i++)
{
if (Store[Key][i][0] == Value && Store[Key][i][1] == 
this)
{
count++;
	Store[Key][i] = tuple(null, null); // Set to null to release 
any references if necessary
	swap(Store[Key][i], Store[Key][max(0, Store[Key].length - 
count)]);

i = i - 1;
}

}
if (count == 1 && Store[Key].length == 1)
{
Store[Key] = null;
Store.remove(Key);
} else
//Store[Key] = 
Store[Key][0..max(0,Store[Key].length-count)];
Store[Key].length = Store[Key].length - count;

Value = null;
Key = null;

}

	public static auto New(TKey k, TValue v, ref Tuple!(TValue, 
SingleStore!(TKey, TValue))[][TKey] s)

{
auto o = new SingleStore!(TKey, TValue)();  // GC
o.Key = k;
o.Value = v;
s[k] ~= tuple(v, o);
o.Store = s;

return o;
}

}


// Creates a static Associative Array that stores multiple values 
per key. The object returned by New can then be used to remove 
the key/value without having to remember specifically them.

public mixin template ObjectStore(TKey, TValue)
{
	// The object store. It is static. Mixin the template into it's 
different types to create different types of stores. All objects 
of that type are then in the same store.
	public static Tuple!(TValue, SingleStore!(TKey, TValue))[][TKey] 
Store;


public static auto New(TKey k, TValue v)
{
auto r =  SingleStore!(TKey, TValue).New(k, v, Store);
return r;
}

}

//alias dg = int delegate(int);
alias dg = string;

public class cMyStore(TKey, TValue)
{
//mixin ObjectStore!(string, dg);
mixin ObjectStore!(string, string); 
}



void main()
{
alias MyStore = cMyStore!(string, string);
auto k = "x";

auto r = &MyStore.Store;

/*
dg d1 = (int x) { return x; };
dg d2 = (int x) { return x; };
dg d3 = d1;
dg d4 = (int x) { return 3*x; };
*/

dg d1 = "a1";
dg d2 = "a2";
dg d3 = "a3";
dg d4 = "a4";



auto s = MyStore.New(k, d1);
writeln(MyStore.Store[k].length);
auto s1 = MyStore.New(k, d2);
writeln(MyStore.Store[k].length);
auto s2 = MyStore.New(k, d3);
writeln(MyStore.Store[k].length);
auto s3 = MyStore.New(k, d4);
writeln(MyStore.Store[k].length);   


//auto x = MyStore.Sto

Re: Nested classes question?

2015-09-15 Thread Adam D. Ruppe via Digitalmars-d-learn

On Wednesday, 16 September 2015 at 01:12:58 UTC, Dave Akers wrote:
When a program exits and D's memory management is cleaning up 
calling all of the ~this's is there a reason it calls the outer 
class's ~this before the inner class's ~this?


All class destructors are called in an undefined order. The 
garbage collector considers the whole object tree to be dead at 
once and just cleans it up in the most convenient order for it.


This means you should not access any member pointers or 
references to things that are managed by the garbage collector in 
a destructor. Manage subobjects manually if you want to destroy 
them in ~this.


Nested classes question?

2015-09-15 Thread Dave Akers via Digitalmars-d-learn
When a program exits and D's memory management is cleaning up 
calling all of the ~this's is there a reason it calls the outer 
class's ~this before the inner class's ~this?


I was recently exploring the possibility of using 
https://github.com/bheads/d-leveldb and the example in the readme 
seg faulted, when digging into it i found out that the outer 
class was being destroyed before the inner causing the database 
to be closed before an iterator for the database was destroyed.


Runtime error when calling a callback in a parallel Task

2015-09-15 Thread BBasile via Digitalmars-d-learn
Under Windows this works fine but under Linux I got a runtime 
error.


this could be reduced to :

---
import std.parallelism;

alias CallBack = void function(void*);

class Foo
{
CallBack clbck;
void* param;
void dotask()
{
// some heavy processing
// tells the caller that some fresh data are available
if(clbck) clbck(param);  // debugger breaks HERE
}

void call()
{
task(&dotask).executeInNewThread;
// returns directly but the caller will get a notif when 
finished

}
}
---

more info about the environment:
- linux i386
- the D program is actually a .so and the main thread is the exe 
that loads this .so.

- If i don't use a Task then the program works **fine**.

Is it possible to achieve this in a cross platform-way ?
How can i get in phase with the main big thread at the end of my 
task ?


Re: Initalizing complex array types or some other problem ;/

2015-09-15 Thread anonymous via Digitalmars-d-learn
On Tuesday 15 September 2015 22:09, Prudence wrote:

> The code below doesn't work.

Please be more specific in how it doesn't work. Mention the error message if 
there is one, or say how the code behaves differently from what you'd 
expect.

Trying to compile the code (after kicking getch out), I get this error:
core.exception.RangeError@test.d(103): Range violation

Line 103 is: writeln(MyStore.Store[k].length);

I can't find where you set Store[k]. Maybe you forgot that or deleted it 
accidentally?

I made a guess and added this line in SingleStore.New:
o.Store[k] ~= tuple(v, o);

It still throws a range error with this. That's because associative arrays 
are a little weird, unfortunately.

An AA is effectively initialized on the first assignment. So (with my 
addition) the first SingleStore.New call initializes the AA. But it only 
initializes o.Store, not the original variable s (i.e. ObjectStore.Store). s 
is left empty.

Possible solutions/workarounds:
* Append to s[k], then assign s to o.Store.
* Initialize ObjectStore.Store in a static constructor:

static this()
{
Store[TKey.init] = [];
Store.remove(TKey.init);
}


Aside: I have no idea what you're doing there, but it looks pretty 
complicated, to the point that I'd guess that it's more complicated than 
necessary.


Re: Checking for Homogeneous Tuples

2015-09-15 Thread Nordlöw via Digitalmars-d-learn

On Tuesday, 15 September 2015 at 17:41:07 UTC, Meta wrote:
I made a pull request for this a long time ago but it was 
rejected.


https://github.com/D-Programming-Language/phobos/pull/1672


I'll try again because it's needed here

https://github.com/D-Programming-Language/phobos/pull/3395

;)


Initalizing complex array types or some other problem ;/

2015-09-15 Thread Prudence via Digitalmars-d-learn
How does one initialize a tuple type and are arrays initialized 
by default?



The code below doesn't work. I recently converted to use a tuple 
and that seemed to have broke it and I don't know why, when I 
changed the New function to use a ref, that made it worse cause 
now the array is all ways null. Debugging doesn't help much 
because, for some reason, VD won't show object information inside 
the mixin ObjectStore.



	alias StoreType = Tuple!(TValue, SingleStore!(TKey, 
TValue))[][TKey];

public static StoreType Store;

I thought using the alias would help but it doesn't. Trying to 
init it in the static this doesn't seem to help me, as I can't 
get it to new propertly. e.g., new StoreType(); fails.


Any ideas?






import std.stdio;
import std.concurrency;



extern (C) int getch();
import std.string;
import std.concurrency;
import core.time;
import core.thread;
import std.container.array;
import std.typecons;







public class SingleStore(TKey, TValue)
{
public TValue Value;
public TKey Key;
public Tuple!(TValue, SingleStore!(TKey, TValue))[][TKey] Store;


public auto Remove()
{
import std.algorithm;
if (Value == null || Key == null) return;
int count = 0;
for(int i = 0; i < Store[Key].length; i++)
{
if (Store[Key][i][0] == Value && Store[Key][i][1] == 
this)
{
count++;
Store[Key][i] = tuple(null, null);
	swap(Store[Key][i], Store[Key][max(0, Store[Key].length - 
count)]);

i = i - 1;
}


}
if (count == 1 && Store[Key].length == 1)
{
Store[Key] = null;
Store.remove(Key);
} else
//Store[Key] = 
Store[Key][0..max(0,Store[Key].length-count)];
Store[Key].length = Store[Key].length - count;

Value = null;
Key = null;

}

	public static auto New(TKey k, TValue v, ref Tuple!(TValue, 
SingleStore!(TKey, TValue))[][TKey] s)

{
auto o = new SingleStore!(TKey, TValue)(k, v);  // GC
o.Store = s;
return o;
}

private this(TKey k, TValue v)
{
Key = k;
Value = v;
}
}


// Creates a static Associative Array that stores multiple values 
per key. The object returned by New can then be used to remove 
the key/value without having to remember them specifically.

public mixin template ObjectStore(TKey, TValue)
{
	alias StoreType = Tuple!(TValue, SingleStore!(TKey, 
TValue))[][TKey];

public static StoreType Store;

public static auto New(TKey k, TValue v)
{
auto r =  SingleStore!(TKey, TValue).New(k, v, Store);
return r;
}
}

alias dg = int delegate(int);

class MyStore
{
mixin ObjectStore!(string, dg);
}

void main()
{

auto k = "x";

dg d1 = (int x) { return x; };
dg d2 = (int x) { return x; };
dg d3 = d1;
dg d4 = (int x) { return 3*x; };

auto s = MyStore.New(k, d1);
writeln(MyStore.Store[k].length);
auto s1 = MyStore.New(k, d2);
writeln(MyStore.Store[k].length);
auto s2 = MyStore.New(k, d3);
writeln(MyStore.Store[k].length);
auto s3 = MyStore.New(k, d4);
writeln(MyStore.Store[k].length);   

s1.Remove();
writeln(MyStore.Store[k].length);
s2.Remove();
writeln(MyStore.Store[k].length);
s.Remove();
writeln(MyStore.Store[k].length);
s3.Remove();



getch();
}


Re: tkd not linking

2015-09-15 Thread Gary Willoughby via Digitalmars-d-learn

On Tuesday, 15 September 2015 at 17:37:40 UTC, karabuta wrote:
I have tried several times to compile tkd using dub but I keep 
getting this message:


Linking...
/usr/bin/ld: cannot find -ltcl
/usr/bin/ld: cannot find -ltk


It looks like the libraries are not installed or not installed 
properly.





Re: Speeding up text file parser (BLAST tabular format)

2015-09-15 Thread Fredrik Boulund via Digitalmars-d-learn

On Tuesday, 15 September 2015 at 18:42:29 UTC, Andrew Brown wrote:
I had some luck building a local copy of llvm in my home 
directory, using a linux version about as old as yours (llvm 
3.5 i used) specifying:


--configure --prefix=/home/andrew/llvm

so make install would install it somewhere I had permissions.

Then I changed the cmake command to:

cmake -L -DLLVM_CONFIG="/home/andrew/llvm/bin/llvm-config" ..

and I got a working install of ldc.

Make yourself a cup of tea while you wait though if you try it, 
llvm was about an hour and a half to compile.




Thanks for your suggestion. I'm amazed by the amount of effort 
you guys put into helping me. Unfortunately the only precompiled 
version of libstdc++ available for the system in question via Red 
Hat repo's is 4.4.7, and compiling llvm from scratch requires at 
least 4.7. I'll be fine using DMD for now as I'm still learning 
more about D :).




Re: Speeding up text file parser (BLAST tabular format)

2015-09-15 Thread Andrew Brown via Digitalmars-d-learn
I had some luck building a local copy of llvm in my home 
directory, using a linux version about as old as yours (llvm 3.5 
i used) specifying:


--configure --prefix=/home/andrew/llvm

so make install would install it somewhere I had permissions.

Then I changed the cmake command to:

cmake -L -DLLVM_CONFIG="/home/andrew/llvm/bin/llvm-config" ..

and I got a working install of ldc.

Make yourself a cup of tea while you wait though if you try it, 
llvm was about an hour and a half to compile.


On Tuesday, 15 September 2015 at 13:49:04 UTC, Fredrik Boulund 
wrote:
On Tuesday, 15 September 2015 at 10:01:30 UTC, John Colvin 
wrote:
try this: 
https://dlangscience.github.io/resources/ldc-0.16.0-a2_glibc2.11.3.tar.xz


Nope, :(

$ ldd ldc2
./ldc2: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not 
found (required by ./ldc2)

linux-vdso.so.1 =>  (0x7fff2ffd8000)
libpthread.so.0 => /lib64/libpthread.so.0 
(0x00318a00)

libdl.so.2 => /lib64/libdl.so.2 (0x00318a40)
libncurses.so.5 => /lib64/libncurses.so.5 
(0x00319bc0)

librt.so.1 => /lib64/librt.so.1 (0x00318a80)
libz.so.1 => /lib64/libz.so.1 (0x00318ac0)
libstdc++.so.6 => /usr/lib64/libstdc++.so.6 
(0x00318dc0)

libm.so.6 => /lib64/libm.so.6 (0x003189c0)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 
(0x00318c00)

libc.so.6 => /lib64/libc.so.6 (0x00318980)
/lib64/ld-linux-x86-64.so.2 (0x00318940)
libtinfo.so.5 => /lib64/libtinfo.so.5 
(0x00319900)


Thanks for trying though!





Re: No -v or -deps for gdc?

2015-09-15 Thread Johannes Pfau via Digitalmars-d-learn
Am Tue, 15 Sep 2015 12:19:34 +
schrieb Atila Neves :

> gdmd supports those options but gdc doesn't. Is that likely to 
> always be the case?
> 
> Atila

gdmd is just a wrapper around gdc. If something is supported by gdmd it
must also be supported by gdc (the exact switch names might differ).

See:
https://github.com/D-Programming-GDC/GDMD/blob/master/dmd-script

Seems like -v maps to -fd-verbose and -deps to -fdeps.


Re: Speeding up text file parser (BLAST tabular format)

2015-09-15 Thread H. S. Teoh via Digitalmars-d-learn
On Tue, Sep 15, 2015 at 08:55:43AM +, Fredrik Boulund via 
Digitalmars-d-learn wrote:
> On Monday, 14 September 2015 at 18:31:38 UTC, H. S. Teoh wrote:
> >I tried implementing a crude version of this (see code below), and
> >found that manually calling GC.collect() even as frequently as once
> >every 5000 loop iterations (for a 500,000 line test input file) still
> >gives about 15% performance improvement over completely disabling the
> >GC.  Since most of the arrays involved here are pretty small, the
> >frequency could be reduced to once every 50,000 iterations and you'd
> >pretty much get the 20% performance boost for free, and still not run
> >out of memory too quickly.
> 
> Interesting, I'll have to go through your code to understand exactly
> what's going on. I also noticed some GC-related stuff high up in my
> profiling, but had no idea what could be done about that. Appreciate
> the suggestions!

It's very simple, actually. Basically you just call GC.disable() at the
beginning of the program to disable automatic collection cycles, then at
period intervals in you manually trigger collections by calling
GC.collect().

The way I implemented it in my test code was to use a global counter
that I decrement once every loop iteration. When the counter reaches
zero, GC.collect() is called, and then the counter is reset to its
original value.  This is encapsulated in the gcTick() function, so that
it's easy to tweak the frequency of the manual collections without
modifying several different places in the code each time.


T

-- 
BREAKFAST.COM halted...Cereal Port Not Responding. -- YHL


Re: foreach(line; f.byLine) produces core.exception.InvalidMemoryOperationError@(0) in 2.067 but not 2.066

2015-09-15 Thread Andrew Brown via Digitalmars-d-learn
Thanks very much for your help, it seemed to work a treat (I hope 
:))! Compiling ldc wasn't too bad, make the changes to 
runtime/phobos/std/stdio.d and then just building as normal was 
no problem. Unittests are passing and it handles that file 
perfectly.


On Tuesday, 15 September 2015 at 16:11:06 UTC, Martin Krejcirik 
wrote:
On Tuesday, 15 September 2015 at 15:28:23 UTC, Andrew Brown 
wrote:
A very naive question: would it be possible in this case to 
backport it into gdc/ldc by copying the pull request and 
building the compiler from source, or would this get me into a 
world of pain?


Cherry-picking should work and merge cleanly. I have done it 
for DMD 2.067. I don't know how difficult it is to recompile 
Phobos and Druntime with LDC/GDC though.





Re: tkd not linking

2015-09-15 Thread Adam D. Ruppe via Digitalmars-d-learn

On Tuesday, 15 September 2015 at 17:37:40 UTC, karabuta wrote:

Just incase, I have install version 8.6 of the Tcl/Tk libraries


Did you get the development version?

sudo apt-get install tk-dev

or possibly

sudo apt-get install tk8.6-dev

should do it. I'm not actually sure if that's required for this 
(I've never used it) but it might be.


What is the problem. And how is tcltk different from tkd in the 
first place?


tkd is just a D wrapper for the library.


Re: Checking for Homogeneous Tuples

2015-09-15 Thread Meta via Digitalmars-d-learn

On Tuesday, 15 September 2015 at 16:54:22 UTC, Nordlöw wrote:
How do I check that all the elements of a std.typecons.Tuple 
all fulfil a specific predicate, in my case all have a specific 
type:


Something like

import std.typecons : isTuple;
enum isTupleOf(T, E) = isTuple!T && is(MAGIC(T, E));


I made a pull request for this a long time ago but it was 
rejected.


https://github.com/D-Programming-Language/phobos/pull/1672


tkd not linking

2015-09-15 Thread karabuta via Digitalmars-d-learn
I have tried several times to compile tkd using dub but I keep 
getting this message:


Linking...
/usr/bin/ld: cannot find -ltcl
/usr/bin/ld: cannot find -ltk
collect2: error: ld returned 1 exit status
--- errorlevel 1
FAIL 
.dub/build/application-debug-linux.posix-x86-dmd_2068-4989C12BA459945625492BF92EAC638A/ tkdapps executable

Error executing command run:
dmd failed with exit code 1.


I use Ubuntu 14.04 x32. Here is the dub.json file content

{
"name": "tkdapps",
"description": "A minimal D application.",
"copyright": "Copyright © 2015, karabuta",
"authors": ["karabuta"],
"dependencies": {
"tcltk": "8.6.5",
"tkd": "1.1.4"
}
}

Just incase, I have install version 8.6 of the Tcl/Tk libraries

What is the problem. And how is tcltk different from tkd in the 
first place?




Re: Checking for Homogeneous Tuples

2015-09-15 Thread Adam D. Ruppe via Digitalmars-d-learn
If it is a tuple of values too, you could just try to form an 
array out of it: `static if (__traits(compiles, [your_tuple]))`. 
But allSatisfy might be better.


For the predicate there, remember it needs to take a template 
argument.


Re: Checking for Homogeneous Tuples

2015-09-15 Thread John Colvin via Digitalmars-d-learn

On Tuesday, 15 September 2015 at 16:54:22 UTC, Nordlöw wrote:
How do I check that all the elements of a std.typecons.Tuple 
all fulfil a specific predicate, in my case all have a specific 
type:


Something like

import std.typecons : isTuple;
enum isTupleOf(T, E) = isTuple!T && is(MAGIC(T, E));


std.meta.allSatisfy!(SomePredicateTemplate, T.Types);


Re-named & Selective Imports

2015-09-15 Thread jmh530 via Digitalmars-d-learn
I combined a re-named import with a selective import and was 
surprised to find that it didn't do what I would have expected. 
In the code below, I would have expected only the "test2" line to 
have compiled, but it turned out that all three of these do. I'm 
guessing the logic is that it imports all of std.stdio, re-names 
it as io, then puts just std.stdio.writeln in the namespace.


import io = std.stdio : writeln;

void main()
{
writeln("test1");
io.writeln("test2");
io.write("test3", '\n');
}

It seems like it might be a bit more intuitive if they had 
disabled selective imports with re-named imports (like with 
static imports). Then you could do a separate selective import. 
This would probably be more clear.


As far as I can tell, if you do selective imports of a function 
with the same name from two different modules, then you have to 
give one a unique name without any module prefix. For instance, I 
tried to import with
import std.stdio : writeln, io.writeln = writeln;, but that 
doesn't work, but something like io_writeln would.





Checking for Homogeneous Tuples

2015-09-15 Thread Nordlöw via Digitalmars-d-learn
How do I check that all the elements of a std.typecons.Tuple all 
fulfil a specific predicate, in my case all have a specific type:


Something like

import std.typecons : isTuple;
enum isTupleOf(T, E) = isTuple!T && is(MAGIC(T, E));




Re: foreach(line; f.byLine) produces core.exception.InvalidMemoryOperationError@(0) in 2.067 but not 2.066

2015-09-15 Thread Martin Krejcirik via Digitalmars-d-learn

On Tuesday, 15 September 2015 at 15:28:23 UTC, Andrew Brown wrote:
A very naive question: would it be possible in this case to 
backport it into gdc/ldc by copying the pull request and 
building the compiler from source, or would this get me into a 
world of pain?


Cherry-picking should work and merge cleanly. I have done it for 
DMD 2.067. I don't know how difficult it is to recompile Phobos 
and Druntime with LDC/GDC though.


Re: Speeding up text file parser (BLAST tabular format)

2015-09-15 Thread John Colvin via Digitalmars-d-learn
On Tuesday, 15 September 2015 at 13:49:04 UTC, Fredrik Boulund 
wrote:
On Tuesday, 15 September 2015 at 10:01:30 UTC, John Colvin 
wrote:

[...]


Nope, :(

[...]


Oh well, worth a try I guess.


Re: foreach(line; f.byLine) produces core.exception.InvalidMemoryOperationError@(0) in 2.067 but not 2.066

2015-09-15 Thread Andrew Brown via Digitalmars-d-learn
On Tuesday, 15 September 2015 at 14:55:42 UTC, Martin Krejcirik 
wrote:

For reference, it was this PR:
https://github.com/D-Programming-Language/phobos/pull/3089
which fixed the same issue for me.


A very naive question: would it be possible in this case to 
backport it into gdc/ldc by copying the pull request and building 
the compiler from source, or would this get me into a world of 
pain?


Re: Compilation error

2015-09-15 Thread Loic via Digitalmars-d-learn
On Tuesday, 15 September 2015 at 12:37:46 UTC, Adam D. Ruppe 
wrote:

On Tuesday, 15 September 2015 at 09:17:26 UTC, Loic wrote:
Error: cannot find source code for runtime library file 
'object.d'


How did you install dmd? The installer exe or the zip both 
should have come with all these files packaged together.


Hello Adam,

I installed with the exe installer, and set my path to 
C:\D\dmd2\windows\bin

Maybe I forget to configure some file(s) ?

Thank you

Loic


Re: foreach(line; f.byLine) produces core.exception.InvalidMemoryOperationError@(0) in 2.067 but not 2.066

2015-09-15 Thread Martin Krejcirik via Digitalmars-d-learn

For reference, it was this PR:
https://github.com/D-Programming-Language/phobos/pull/3089
which fixed the same issue for me.



Re: foreach(line; f.byLine) produces core.exception.InvalidMemoryOperationError@(0) in 2.067 but not 2.066

2015-09-15 Thread Andrew Brown via Digitalmars-d-learn

On Tuesday, 15 September 2015 at 14:19:13 UTC, Daniel Kozák wrote:


Which OS?


It's CentOS release 6.5 (Final), I tried dmd 2.068.1 and the 
problem has disappeared. Thanks very much for the advice, I can 
stick to old gdc for speed until ldc catches up to 2.068.


Best

Andrew


Re: foreach(line; f.byLine) produces core.exception.InvalidMemoryOperationError@(0) in 2.067 but not 2.066

2015-09-15 Thread Martin Krejcirik via Digitalmars-d-learn

On Tuesday, 15 September 2015 at 13:56:37 UTC, Andrwe Brown wrote:
I'm trying to read a file line by line, and I get a 
core.exception.InvalidMemoryOperationError@(0), even after


https://issues.dlang.org/show_bug.cgi?id=13856

Try DMD 2.068, it has got fixed byLine implementation.




Re: foreach(line; f.byLine) produces core.exception.InvalidMemoryOperationError@(0) in 2.067 but not 2.066

2015-09-15 Thread Daniel Kozák via Digitalmars-d-learn

On Tue, 15 Sep 2015 13:56:36 +
Andrwe Brown via Digitalmars-d-learn
 wrote:

> Hi,
> 
> I'm trying to read a file line by line, and I get a 
> core.exception.InvalidMemoryOperationError@(0), even after 
> reducing the program to:
> 
> import std.stdio;
> 
> void main()
> {
>File f = File("testfile");
>foreach(line; f.byLine)
>{
>}
> }
> 
> The file is a simple table of ascii characters, 811 columns and 
> it fails on the second line. Taking any subset of the columns and 
> the program runs fine so I don't think it can by any particular 
> file corruption.
> 
> In this simple example I can get it to work by changing:
> 
> foreach(line; f.byLine)
> 
> to
> 
> foreach(char[] line; f.byLine)
> 
> but in my more complicated program this still fails with the same 
> error:
> 
> foreach (char[] lineVar; inFile.byLine)
> {
>lineVar.split.indexed(places).joiner("\t").writeln;
> }
> 
> (as does the range version I originally wrote:
> inFile.byLine.map!(a => 
> a.split.indexed(places).joiner("\t")).joiner("\n").writeln;)
> 
> Is this a bug, gdc on version 2.066 seems to have no problems 
> with it? I'd be happy to fill in a bug report, but I can't share 
> the file as it's sensitive genetic data and I haven't been able 
> to reduce it to anything innocuous.
> 
> This has me very puzzled, any suggestions would be much 
> appreciated.
> 
> Thanks very much
> 
> Andrew

Which OS?


Re: Adjacent Pairs Range

2015-09-15 Thread Sebastiaan Koppe via Digitalmars-d-learn

On Monday, 14 September 2015 at 10:45:52 UTC, Per Nordlöw wrote:
On Monday, 14 September 2015 at 05:37:05 UTC, Sebastiaan Koppe 
wrote:

What about using zip and a slice?


Slicing requires a RandomAccessRange (Array). This is too 
restrictive. We want to change operations such as 
adjacentTuples with for example map and reduce without the need 
for temporary copies of the whole range. This is the thing 
about D's standard library.


Read up on D's range concepts.


dropOne then.

I saw your adjacentTuples. Two questions:

a) can't you use a ringbuffer instead of copy when N > 2?
b) shouldn't front() return a range over that ringbuffer?


foreach(line; f.byLine) produces core.exception.InvalidMemoryOperationError@(0) in 2.067 but not 2.066

2015-09-15 Thread Andrwe Brown via Digitalmars-d-learn

Hi,

I'm trying to read a file line by line, and I get a 
core.exception.InvalidMemoryOperationError@(0), even after 
reducing the program to:


import std.stdio;

void main()
{
  File f = File("testfile");
  foreach(line; f.byLine)
  {
  }
}

The file is a simple table of ascii characters, 811 columns and 
it fails on the second line. Taking any subset of the columns and 
the program runs fine so I don't think it can by any particular 
file corruption.


In this simple example I can get it to work by changing:

foreach(line; f.byLine)

to

foreach(char[] line; f.byLine)

but in my more complicated program this still fails with the same 
error:


foreach (char[] lineVar; inFile.byLine)
{
  lineVar.split.indexed(places).joiner("\t").writeln;
}

(as does the range version I originally wrote:
inFile.byLine.map!(a => 
a.split.indexed(places).joiner("\t")).joiner("\n").writeln;)


Is this a bug, gdc on version 2.066 seems to have no problems 
with it? I'd be happy to fill in a bug report, but I can't share 
the file as it's sensitive genetic data and I haven't been able 
to reduce it to anything innocuous.


This has me very puzzled, any suggestions would be much 
appreciated.


Thanks very much

Andrew


Re: Speeding up text file parser (BLAST tabular format)

2015-09-15 Thread Fredrik Boulund via Digitalmars-d-learn

On Tuesday, 15 September 2015 at 10:01:30 UTC, John Colvin wrote:
try this: 
https://dlangscience.github.io/resources/ldc-0.16.0-a2_glibc2.11.3.tar.xz


Nope, :(

$ ldd ldc2
./ldc2: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not 
found (required by ./ldc2)

linux-vdso.so.1 =>  (0x7fff2ffd8000)
libpthread.so.0 => /lib64/libpthread.so.0 
(0x00318a00)

libdl.so.2 => /lib64/libdl.so.2 (0x00318a40)
libncurses.so.5 => /lib64/libncurses.so.5 
(0x00319bc0)

librt.so.1 => /lib64/librt.so.1 (0x00318a80)
libz.so.1 => /lib64/libz.so.1 (0x00318ac0)
libstdc++.so.6 => /usr/lib64/libstdc++.so.6 
(0x00318dc0)

libm.so.6 => /lib64/libm.so.6 (0x003189c0)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00318c00)
libc.so.6 => /lib64/libc.so.6 (0x00318980)
/lib64/ld-linux-x86-64.so.2 (0x00318940)
libtinfo.so.5 => /lib64/libtinfo.so.5 (0x00319900)

Thanks for trying though!


Re: Speeding up text file parser (BLAST tabular format)

2015-09-15 Thread John Colvin via Digitalmars-d-learn

On Tuesday, 15 September 2015 at 13:01:06 UTC, Kagamin wrote:
On Tuesday, 15 September 2015 at 09:19:29 UTC, John Colvin 
wrote:
It provides you only one char at a time instead of a whole 
line. It will be quite constraining for your code if not 
mind-bending.


http://dlang.org/phobos/std_string.html#.lineSplitter

File(fileName).byChunk(chunkSize).map!"cast(char[])a".joiner().lineSplitter()


lineSplitter doesn't work with input ranges.


Ugh


Re: Speeding up text file parser (BLAST tabular format)

2015-09-15 Thread Kagamin via Digitalmars-d-learn

On Tuesday, 15 September 2015 at 09:19:29 UTC, John Colvin wrote:
It provides you only one char at a time instead of a whole 
line. It will be quite constraining for your code if not 
mind-bending.


http://dlang.org/phobos/std_string.html#.lineSplitter

File(fileName).byChunk(chunkSize).map!"cast(char[])a".joiner().lineSplitter()


lineSplitter doesn't work with input ranges.


Re: Compilation error

2015-09-15 Thread Adam D. Ruppe via Digitalmars-d-learn

On Tuesday, 15 September 2015 at 09:17:26 UTC, Loic wrote:
Error: cannot find source code for runtime library file 
'object.d'


How did you install dmd? The installer exe or the zip both should 
have come with all these files packaged together.


No -v or -deps for gdc?

2015-09-15 Thread Atila Neves via Digitalmars-d-learn
gdmd supports those options but gdc doesn't. Is that likely to 
always be the case?


Atila


Re: alias for regular expressions

2015-09-15 Thread Thunderbird via Digitalmars-d-learn

On Monday, 14 September 2015 at 07:11:38 UTC, Meta wrote:

On Sunday, 13 September 2015 at 19:52:20 UTC, Thunderbird wrote:

Thanks for your quick reply :)


To expand on that, alias in D is nothing like the C macro 
preprocessor. D specifically disallows tricks like `#define 
true false`. In D, aliases are just another name for a symbol. 
Ex:


struct Test
{
int n;
float f;
}

alias IntFloat = Test; //IntFloat is just another name for Test

Test t = Test(0, 1.0);
IntFloat inf = IntFloat(0, 1.0);

//Test and IntFloat are NOT separate types.
//Use std.typecons.Typedef for that
assert(t == inf);
assert(is(typeof(t) == typeof(inf));


Thanks for elaborating!


Re: Massive linker error after upgrade to DMD 2.068.1-1

2015-09-15 Thread rcorre via Digitalmars-d-learn
On Sunday, 13 September 2015 at 00:21:22 UTC, Nicholas Wilson 
wrote:


there was talk of adding symbol name compression some time ago 
( to reduce lib size ( and maybe make ddemangle not fail on 
long syms?). This might be the cause of your problems i.e. the 
new compiler emitting references to compressed names but built 
with the old compiler using the not compressed ones. Try 
rebuilding everything and see if that fixes it.


I tried deleting ~/.dub and the .dub folder in every referenced 
project and rebuilding, but unfortunately no luck.


Re: Speeding up text file parser (BLAST tabular format)

2015-09-15 Thread John Colvin via Digitalmars-d-learn
On Tuesday, 15 September 2015 at 08:45:00 UTC, Fredrik Boulund 
wrote:

On Monday, 14 September 2015 at 15:04:12 UTC, John Colvin wrote:

[...]


Thanks for the offer, but don't go out of your way for my sake. 
Maybe I'll just build this in a clean environment instead of on 
my work computer to get rid of all the hassle. The Red Hat 
llvm-devel packages are broken, dependent on libffi-devel which 
is unavailable. Getting the build environment up to speed on my 
main machine would take me a lot more time than I have right 
now.


Tried building LDC from scratch but it fails because of missing 
LLVM components, despite having LLVM 3.4.2 installed (though 
lacking devel components).


try this: 
https://dlangscience.github.io/resources/ldc-0.16.0-a2_glibc2.11.3.tar.xz


Re: Speeding up text file parser (BLAST tabular format)

2015-09-15 Thread John Colvin via Digitalmars-d-learn

On Tuesday, 15 September 2015 at 09:09:00 UTC, Kagamin wrote:
On Tuesday, 15 September 2015 at 08:53:37 UTC, Fredrik Boulund 
wrote:

my favourite for streaming a file:
enum chunkSize = 4096;
File(fileName).byChunk(chunkSize).map!"cast(char[])a".joiner()


Is this an efficient way of reading this type of file? What 
should one keep in mind when choosing chunkSize?


reasonably efficient, yes. See http://stackoverflow.com/a/237495 
for a discussion of chunk sizing when streaming a file.


It provides you only one char at a time instead of a whole 
line. It will be quite constraining for your code if not 
mind-bending.


http://dlang.org/phobos/std_string.html#.lineSplitter

File(fileName).byChunk(chunkSize).map!"cast(char[])a".joiner().lineSplitter()


Compilation error

2015-09-15 Thread Loic via Digitalmars-d-learn


Hello,

I hope it's the good place to ask my question.

I'am trying an hello world program in D, unfortunately the 
compilation, doesn't work, and found nothing on google.


when I do : dmd Hello.d, the error returned is

Error: cannot find source code for runtime library file 'object.d'
dmd might not be correctly installed. Run 'dmd -man' for 
installation instructions.

config file: C:\D\dmd2\windows\bin\sc.ini
Specify path to file 'object.d' with -I switch

My machine is Windows 7 64 bits DMD2 and DMC are installed.

Thank you

Loic


Re: Speeding up text file parser (BLAST tabular format)

2015-09-15 Thread Kagamin via Digitalmars-d-learn
On Tuesday, 15 September 2015 at 08:53:37 UTC, Fredrik Boulund 
wrote:

my favourite for streaming a file:
enum chunkSize = 4096;
File(fileName).byChunk(chunkSize).map!"cast(char[])a".joiner()


Is this an efficient way of reading this type of file? What 
should one keep in mind when choosing chunkSize?


It provides you only one char at a time instead of a whole line. 
It will be quite constraining for your code if not mind-bending.


Re: Speeding up text file parser (BLAST tabular format)

2015-09-15 Thread Rikki Cattermole via Digitalmars-d-learn

On 15/09/15 9:00 PM, Kagamin wrote:

On Tuesday, 15 September 2015 at 08:51:02 UTC, Fredrik Boulund wrote:

Using char[] all around might be a good idea, but it doesn't seem like
the string conversions are really that taxing. What are the arguments
for working on char[] arrays rather than strings?


No, casting to string would be incorrect here because the line buffer is
reused, your original usage of to!string is correct.


I made the assumption it wasn't allocating. Ehhh.


Re: Speeding up text file parser (BLAST tabular format)

2015-09-15 Thread Kagamin via Digitalmars-d-learn
On Tuesday, 15 September 2015 at 08:51:02 UTC, Fredrik Boulund 
wrote:
Using char[] all around might be a good idea, but it doesn't 
seem like the string conversions are really that taxing. What 
are the arguments for working on char[] arrays rather than 
strings?


No, casting to string would be incorrect here because the line 
buffer is reused, your original usage of to!string is correct.


Re: Speeding up text file parser (BLAST tabular format)

2015-09-15 Thread Fredrik Boulund via Digitalmars-d-learn

On Monday, 14 September 2015 at 18:31:38 UTC, H. S. Teoh wrote:
I tried implementing a crude version of this (see code below), 
and found that manually calling GC.collect() even as frequently 
as once every 5000 loop iterations (for a 500,000 line test 
input file) still gives about 15% performance improvement over 
completely disabling the GC.  Since most of the arrays involved 
here are pretty small, the frequency could be reduced to once 
every 50,000 iterations and you'd pretty much get the 20% 
performance boost for free, and still not run out of memory too 
quickly.


Interesting, I'll have to go through your code to understand 
exactly what's going on. I also noticed some GC-related stuff 
high up in my profiling, but had no idea what could be done about 
that. Appreciate the suggestions!




Re: Speeding up text file parser (BLAST tabular format)

2015-09-15 Thread Fredrik Boulund via Digitalmars-d-learn
On Monday, 14 September 2015 at 16:33:23 UTC, Rikki Cattermole 
wrote:


A lot of this hasn't been covered I believe.

http://dpaste.dzfl.pl/f7ab2915c3e1

1) You don't need to convert char[] to string via to. No. Too 
much. Cast it.
2) You don't need byKey, use foreach key, value syntax. That 
way you won't go around modifying things unnecessarily.


Ok, I disabled GC + reserved a bunch of memory. It probably 
won't help much actually. In fact may make it fail so keep that 
in mind.


Humm what else.

I'm worried about that first foreach. I don't think it needs to 
exist as it does. I believe an input range would be far better. 
Use a buffer to store the Hit[]'s. Have a subset per set of 
them.


If the first foreach is an input range, then things become 
slightly easier in the second. Now you can turn that into it's 
own input range.
Also that .array usage concerns me. Many an allocation there! 
Hence why the input range should be the return from it.


The last foreach, is lets assume dummy. Keep in mind, stdout is 
expensive here. DO NOT USE. If you must buffer output then do 
it large quantities.



Based upon what I can see, you are definitely not able to use 
your cpu's to the max. There is no way that is the limiting 
factor here. Maybe your usage of a core is. But not the cpu's 
itself.


The thing is, you cannot use multiple threads on that first 
foreach loop to speed things up. No. That needs to happen all 
on one thread.
Instead after that thread you need to push the result into 
another.


Perhaps, per thread one lock (mutex) + buffer for hits. Go 
round robin over all the threads. If mutex is still locked, 
you'll need to wait. In this situation a locked mutex means all 
you worker threads are working. So you can't do anything more 
(anyway).


Of course after all this, the HDD may still be getting hit too 
hard. In which case I would recommend you memory mapping it. 
Which should allow the OS to more efficiently handle reading it 
into memory. But you'll need to rework .byLine for that.



Wow that was a lot at 4:30am! So don't take it too seriously. 
I'm sure somebody else will rip that to shreds!


Thanks for your suggestions! That sure is a lot of details. I'll 
have to go through them carefully to understand what to do with 
all this. Going multithreaded sounds fun but would  effectively 
kill of all of my spare time, so I might have to skip that. :)


Using char[] all around might be a good idea, but it doesn't seem 
like the string conversions are really that taxing. What are the 
arguments for working on char[] arrays rather than strings?


I'm aware that printing output like that is a performance killer, 
but it's not supposed to write anything in the final program. 
It's just there for me to be able to compare the results to my 
Python code.


Re: Speeding up text file parser (BLAST tabular format)

2015-09-15 Thread Fredrik Boulund via Digitalmars-d-learn

On Monday, 14 September 2015 at 18:08:31 UTC, John Colvin wrote:
On Monday, 14 September 2015 at 17:51:43 UTC, CraigDillabaugh 
wrote:
On Monday, 14 September 2015 at 12:30:21 UTC, Fredrik Boulund 
wrote:

[...]


I am going to go off the beaten path here.  If you really want 
speed
for a file like this one way of getting that is to read the 
file
in as a single large binary array of ubytes (or in blocks if 
its too big)
and parse the lines yourself. Should be fairly easy with D's 
array slicing.


my favourite for streaming a file:
enum chunkSize = 4096;
File(fileName).byChunk(chunkSize).map!"cast(char[])a".joiner()


Is this an efficient way of reading this type of file? What 
should one keep in mind when choosing chunkSize?


Re: Speeding up text file parser (BLAST tabular format)

2015-09-15 Thread Fredrik Boulund via Digitalmars-d-learn
On Monday, 14 September 2015 at 16:13:14 UTC, Edwin van Leeuwen 
wrote:
See this link for clarification on what the columns/numbers in 
the profile file mean

http://forum.dlang.org/post/f9gjmo$2gce$1...@digitalmars.com

It is still difficult to parse though. I myself often use 
sysprof (only available on linux), which automatically ranks by 
time spent.


Thanks for the link. I read up on what everything means, but I 
think the problem isn't finding what consumes the most time, the 
problem is me not knowing the standard library well enough to 
translate the largest consumers to actual parts of my code :).


Re: Speeding up text file parser (BLAST tabular format)

2015-09-15 Thread Fredrik Boulund via Digitalmars-d-learn

On Monday, 14 September 2015 at 15:04:12 UTC, John Colvin wrote:


I've had nothing but trouble when using different versions of 
libc. It would be easier to do this instead: 
http://wiki.dlang.org/Building_LDC_from_source


I'm running a build of LDC git HEAD right now on an old server 
with 2.11, I'll upload the result somewhere once it's done if 
it might be useful


Thanks for the offer, but don't go out of your way for my sake. 
Maybe I'll just build this in a clean environment instead of on 
my work computer to get rid of all the hassle. The Red Hat 
llvm-devel packages are broken, dependent on libffi-devel which 
is unavailable. Getting the build environment up to speed on my 
main machine would take me a lot more time than I have right now.


Tried building LDC from scratch but it fails because of missing 
LLVM components, despite having LLVM 3.4.2 installed (though 
lacking devel components).