Multi-threaded sorting of text file

2020-01-03 Thread MGW via Digitalmars-d-learn

Need help:
There' s a large text file (hundreds of thousands of lines).
The structure is as follows:
2345|wedwededwedwedwe ..
872625|rfrferwewweww .
23|rergrferfefer 


It is necessary to sort this file by the first field having 
received:

23|rergrferfefer...
2345|wedwededwedwedwe...
872625|rfrferwewweww...

There are also N CPU (from 4 to 8) and 16 Gb of Memory. Necessary
come up with an algorithm in D for fast sorting using 
multithreading.





Re: Using tasks without GC?

2020-01-03 Thread Elronnd via Digitalmars-d-learn
You can control when the gc runs.  So if you know the allocations 
are small enough that they won't OOM, then you can say 
GC.disable, and it straight up won't run at all.  But you can 
manually run a collection cycle (during a loading screen or 
whatever) with GC.collect.  See 
http://dpldocs.info/experimental-docs/core.memory.GC.html


Re: Using tasks without GC?

2020-01-03 Thread H. S. Teoh via Digitalmars-d-learn
On Sat, Jan 04, 2020 at 12:51:15AM +, Chris Katko via Digitalmars-d-learn 
wrote:
[...]
> I'm looking through D's parallelism module and the docs state,
> up-front:
> 
>  >Creates a Task on the GC heap that calls an alias.
> 
> The modern, scalable way to make a parallel game engine uses tasks.
> (as opposed to functional decomposition where 1 thread is networking 1
> thread is physics, etc.) That requires making LOTS of tasks (_per
> frame_!) and dispatching them. And a 60 FPS frametime is... 16 ms or
> less.
[...]

I don't know the inner workings of std.parallelism well enough to answer
your question directly, but couldn't you just spawn a bunch of workers
that can be assigned tasks? So you only allocate up-front when first
spawning the workers, but during the main loop you have a fixed number
of workers that can receive and execute tasks.  So you don't have to
allocate at all after the initial up-front allocation.

Generally, doing a lot of small GC allocations is bad for performance. I
would avoid doing that, and use a fixed number of workers that you
create, say at the beginning of every level or something, and thereafter
you never deallocate them, just leave them idle if there are no tasks,
otherwise have some dispatcher to assign tasks to them.


T

-- 
Latin's a dead language, as dead as can be; it killed off all the Romans, and 
now it's killing me! -- Schoolboy


Using tasks without GC?

2020-01-03 Thread Chris Katko via Digitalmars-d-learn
When I program, it's usually videogame ideas. That implies a 
soft, real-time requirement. In general, that requires the mantra 
"allocations are evil, use object pools whenever possible." 
[storing data in static arrays and 'deleting' is usually just 
marking an entry as is_deleted=true and re-using "dead" ones.]


I'm looking through D's parallelism module and the docs state, 
up-front:


 >Creates a Task on the GC heap that calls an alias.

The modern, scalable way to make a parallel game engine uses 
tasks. (as opposed to functional decomposition where 1 thread is 
networking 1 thread is physics, etc.) That requires making LOTS 
of tasks (_per frame_!) and dispatching them. And a 60 FPS 
frametime is... 16 ms or less.


So my question is: Has anyone done any analysis over how 
"dangerous" it is to use GC'd tasks for _small_ tasks (in terms 
of milliseconds)? Is the GC going to fire off all the time and 
send jitter off the charts? Because while freeze-the-world for 20 
milliseconds would probably be unnoticable for many business 
apps, it would completely break a videogame.


I wonder how difficult it would be to either modify the existing 
parallel task codebase (or, write my own?) to use static pools 
instead. Allocate once an array of "MAX_NUM_TASKS" tasks (eating 
the minor memory hit) to prevent touching any allocation. [Even 
if it wasn't GC, allocating every frame in say, C++, is 
dangerous. malloc/new is slow and subject to fragmentation and 
permissions checks.]


Any advice, thoughts? Thanks,
--Chris Katko



Re: How to use ResizerWidget in Dlangui app..?

2020-01-03 Thread ShadoLight via Digitalmars-d-learn

On Thursday, 2 January 2020 at 05:24:33 UTC, Rémy Mouëza wrote:

On Monday, 30 December 2019 at 23:32:37 UTC, ShadoLight wrote:

Hi,

I suspect I'm missing something obvious, but ResizerWidget is 
not working for me on Windows - it shows the 'dragging'-cursor 
when hovering the mouse on the ResizerWidget, but dragging 
with the left mouse button does nothing.




I ran into the same issue. The resizeEvent callback is not 
implemented yet. Below is my custom implementation.



/** A completed resizer widget.
  As of 2016-12-30, the ResizerWidget does not work out of the 
box.

  This class implement the missing piece.
 */
class Resizer : ResizerWidget {

/// Default initialization.
this () {
super ();
initResizeCb ();
}

/// Create with ID parameter.
this (string ID, Orientation orient = Orientation.Vertical) 
{

super (ID, orient);
initResizeCb ();
}

/// Initialize the resize on drag behaviour callback.
protected void initResizeCb () {
this.resizeEvent
= (ResizerWidget source,
   ResizerEventType event,
   int currentPosition)
{
if (event != ResizerEventType.Dragging) {
return;
}

if (_orientation == Orientation.Horizontal) {
auto delta = _previousWidget.width - 
currentPosition;
auto pw= max (0, _previousWidget.width - 
delta);
auto mw= max (0, _nextWidget.width + 
delta);


_previousWidget
.minWidth (pw)
.maxWidth (pw);

_nextWidget
.minWidth (mw)
.maxWidth (mw);
}
else if (_orientation == Orientation.Vertical) {
auto delta = _previousWidget.height - 
currentPosition;
auto pw= max (0, _previousWidget.height - 
delta);
auto mw= max (0, _nextWidget.height + 
delta);


_previousWidget
.minHeight (pw)
.maxHeight (pw);

_nextWidget
.minHeight (mw)
.maxHeight (mw);
}

parent.requestLayout ();
};
}
}


Remy, just a quick note to say thank you again!

I had to make a small adjustment to make the Resizer class 
completely generic for all cases, but I can only guess you did 
not need this as your use case probably had the ResizerWidget 
parent as the outermost widget.


The issue is that the currentPosition argument in the resizeEvent 
handler is in terms of the application window, and not in terms 
of the ResizerWidget parent. So, if you have another widget above 
the ResizerWidget parent (in the Orientation.Vertical case) or to 
the left of the ResizerWidget parent (in the 
Orientation.Horizontal case), you have a small offset issue. The 
same applies if you have margins or padding on the ResizerWidget 
parent (or any of its parent(s)).


Anyway, the fix is quite trivial:

/// Initialize the resize on drag behaviour callback.
protected void initResizeCb () {
this.resizeEvent
= delegate(ResizerWidget source,
   ResizerEventType event,
   int currentPosition)
{
if (event != ResizerEventType.Dragging) {
return;
}

int delta;
int pw;
int mw;
int localPos;
if (_orientation == Orientation.Horizontal) {
localPos = currentPosition - parent.left;
delta = _previousWidget.width - localPos;
pw= max (0, _previousWidget.width - delta);
mw= max (0, _nextWidget.width + delta);

_previousWidget
.minWidth (pw)
.maxWidth (pw);

_nextWidget
.minWidth (mw)
.maxWidth (mw);
}
else if (_orientation == Orientation.Vertical) {
localPos = currentPosition - parent.top;
delta = _previousWidget.height - localPos;
pw= max (0, _previousWidget.height - delta);
mw= max (0, _nextWidget.height + delta);

_previousWidget
.minHeight (pw)
.maxHeight (pw);

_nextWidget
.minHeight (mw)
.maxHeight (mw);
}

parent.requestLayout ();
};
}
}