Re: tanya library 0.2.0

2017-02-22 Thread Eugene Wissner via Digitalmars-d-announce

On Wednesday, 22 February 2017 at 13:00:19 UTC, Kagamin wrote:
On Sunday, 19 February 2017 at 11:41:44 UTC, Eugene Wissner 
wrote:
realloc() can move memory and if an object of type A has 
references to other objects in the array, the objects will be 
corrupted. "A" should be a POD-type. Otherwise you have to 
allocate new memory, initialize it, copy the objects by one 
and deallocate the old memory. Now there is 
IAllocator.expand().


What's the difference between realloc+postblit and copy one by 
one? Postblit is called only after copy, it's not a constructor.


The difference is that realloc will deallocate the old memory 
before postblit.

For the sake of example:

struct Sample
{
Sample* sample;

this(this)
{
sample.sample = 
}
}

void main()
{
auto s = new Sample[2];

s[0].sample = [1];
s[1].sample = [0];
}

Now suppose s[0] has the address 7fc8d2717000, s[1]: 7fc8d2717008.
Realloc cannot reallocate in place and after the call to realloc 
s[0] has the address 7f27771e and s[1]: 7f27771e. It is 
too late to call the postblit here, Sample.sample refers to 
invalid memory.


Re: tanya library 0.2.0

2017-02-22 Thread Kagamin via Digitalmars-d-announce

On Sunday, 19 February 2017 at 11:41:44 UTC, Eugene Wissner wrote:
realloc() can move memory and if an object of type A has 
references to other objects in the array, the objects will be 
corrupted. "A" should be a POD-type. Otherwise you have to 
allocate new memory, initialize it, copy the objects by one and 
deallocate the old memory. Now there is IAllocator.expand().


What's the difference between realloc+postblit and copy one by 
one? Postblit is called only after copy, it's not a constructor.


Re: tanya library 0.2.0

2017-02-19 Thread Eugene Wissner via Digitalmars-d-announce

On Saturday, 18 February 2017 at 23:08:52 UTC, Seb wrote:

Do you know about eventcore [1] (event loop abstraction)?
I hope that this will soon be submitted to Phobos as everyone 
seems to be brewing their own solution and I am _not_ looking 
forward to manage to find a way to run five event loops when I 
want to use fix different libraries.


[1] https://github.com/vibe-d/eventcore

Yes. The event loop is the oldest part of tanya. At the time I 
began to work on it, eventcore was very young and got very seldom 
updates. I only heard it is the new fast and native event loop 
for the new generation of vibe.d. There was libasync aswell which 
was written as a native replacement for libevent, but was never 
adopted more than just as an option; instead eventcore was 
written from the ground up. Botan suffered an even worse fate. So 
I wasn't sure what direction eventcore would take. No of the 
event loops, I was aware of, had IOCP support on Windows. For me 
windows wasn't that important, but I implemented it to ensure 
that the same API can work with both, Posix-style events and the 
asynchronous Windows completion ports; and I could really fix 
some design flaws implementing IOCP. I don't know if I really 
want to have such an abstraction included in phobos, since there 
are bindings for C event loops and a few native ones, and maybe 
they will continue to exist, regardless whether there is 
something in Phobos or not. But yes, if Phobos would have 
something for asynchronous programming, I would take a look into 
it again.


However I would be very interested in hearing your reasons for 
not using it.
Keep in mind that std.experimental.allocator is supposed to be 
a general purpose library exactly to avoid everyone writing 
their own, incompatible Allocators ;-)
So please raise your voice as long as it's still in 
experimental!


[2] https://github.com/bloomberg/bde/wiki/BDE-Allocator-model


Thanks for the question. I had to describe it from the beginning 
or in the README, but didn't want to blow my message up.


BDE allocators implement an interface, other allocators normally 
not. It is the main point. It is not about a concrete interface. 
Phobos allocators are structs, they can't implement anything and 
there is a CAllocatorImpl, that implements IAllocator and wraps 
other allocators, so structs can behave as if they would 
implement IAllocator too. But these structs can be used directly 
without CallocatorImpl in the templated code.

So if you have a container, you can do:
auto arr = Array!(int, Mallocator)(Mallocator.instance);

EMSIContainers do that and it is the way Phobos allocators and 
containers will go. 
(https://github.com/economicmodeling/containers)


If you don't want to template your code or you use phobos default 
allocator, you do the following (because theAllocator should have 
some type):

theAllocator = allocatorObject(Mallocator.instance);


Contras of my allocators:
1) they are slower (because of virtual function calls)
2) If the allocator is passed as template parameter, functions 
that rely on the allocator can infer the attributes like @safe, 
@nogc, pure (yes, there was a discussion to make malloc pure).


Pros:
1) "is-a" relation. Mallocator "is-a" allocator, MmapAllocator 
"is-a" allocator. In your code if you use allocators as a 
template parameter (the faster way), you can't rely that the 
allocator implements some method. So your code will look like:


static if (allocator has method "expand")
{
if (allocator.expand(arr, 8))
{
}
}

2) Less code bloat. With phobos allocators you would make as much 
as possible to templates.


3) I see absolute no reason, why Array!(int, Mallocator) and 
Array!(int, GCAllocator) has different types. Do you want a 
function that accepts an array:


void myFunc(Array!int arr);

you have to make a template function from it (with a long "if 
check" that ensures that an Array!int is passed). Code bloat.


4) It just works everywhere. Phobos allocators are structs but 
there is still IAllocator interface, because the model isn't 
universal enough.


5) I started by implementing IAllocator and the reason I dropped 
it then, is IAllocator.expand. If you have an array of type A 
where A is a struct with a postblit constructor, you can't just 
call realloc() to resize the array. realloc() can move memory and 
if an object of type A has references to other objects in the 
array, the objects will be corrupted. "A" should be a POD-type. 
Otherwise you have to allocate new memory, initialize it, copy 
the objects by one and deallocate the old memory. Now there is 
IAllocator.expand(). If IAllocator.expand() can expand the memory 
in place, everything is fine, you haven't to copy the objects 
because the addresses don't change. But libc realloc for example 
doesn't guarantee that the address doesn't change if you shrink 
the memory. But IAllocator doesn't have a shrink() method.
And anyway I find this separation expand/shrink or 

Re: tanya library 0.2.0

2017-02-18 Thread Seb via Digitalmars-d-announce


On Saturday, 18 February 2017 at 15:51:59 UTC, Eugene Wissner 
wrote:

It isn't really a release announce, maybe a pre-release.

tanya is a general purpose library, used mostly for networking 
by me. It is an attempt to develop an alternative memory model 
for D; 100% of the library are usable in @nogc code.


Sounds really cool!


tanya contains an event loop, containers, an URL parsing 
routine and alternative TCP sockets implementation, multiple 
precision integer. The library is cross plattform, but not 
thread-safe yet.


Do you know about eventcore [1] (event loop abstraction)?
I hope that this will soon be submitted to Phobos as everyone 
seems to be brewing their own solution and I am _not_ looking 
forward to manage to find a way to run five event loops when I 
want to use fix different libraries.


[1] https://github.com/vibe-d/eventcore

The memory management is based on allocators. The allocators 
are one-way compatible with std.experimental.allocator, it 
means my allocators can be used with 
std.exeperimental.allocator but the std.experimental.allocator 
isn't usable with my lbirary (though it is pretty 
straightforward to write a wrapper for phobos allocators). 
tanya's allocators follow Bloomberg Allocator Model, see 
discussions on BDE Allocator Model for pro and contra.


From [2] and your API I get that the BDE API is:

virtual void* allocate(size_type size)
virtual void deallocate(void *address)

whereas you extended this by:

   int alignment()
   void[] allocate(in size_t size)
   bool deallocate(void[] p)
   bool reallocate(ref void[] p, in size_t size)
   bool reallocateInPlace(ref void[] p, in size_t size)

Well obviously the std.experimental.allocator protocol is more 
complicated:


uint alignment()
void[] allocate(size_t, TypeInfo ti = null)
void[] alignedAllocate(size_t n, uint a)
void[] allocateAll()
bool expand(ref void[], size_t);
bool reallocate(ref void[], size_t);
bool alignedReallocate(ref void[] b, size_t size, uint 
alignment);

Ternary owns(void[] b);
Ternary resolveInternalPointer(void* p, ref void[] result);
bool deallocate(void[] b);
bool deallocateAll();
Ternary empty();

However I would be very interested in hearing your reasons for 
not using it.
Keep in mind that std.experimental.allocator is supposed to be a 
general purpose library exactly to avoid everyone writing their 
own, incompatible Allocators ;-)

So please raise your voice as long as it's still in experimental!

[2] https://github.com/bloomberg/bde/wiki/BDE-Allocator-model