Re: A betterC modular standard library?

2016-12-20 Thread Mike via Digitalmars-d

On Tuesday, 20 December 2016 at 12:49:53 UTC, Walter Bright wrote:

I don't propose forcing anyone to use Druntime.


Perhaps this is not part of Ilya's concern, but druntime is 
required to get a build.


  //test.d-
  module test;

  void main() { }

  dmd -m64 -defaultlib= -debuglib= -conf= test.d

  Error: cannot find source code for runtime library file 
'object.d'
   dmd might not be correctly installed. Run 'dmd -man' for 
installation

   instructions.
   config file: (null)
  Specify path to file 'object.d' with -I switch


There's nothing here that really requires druntime, unless you 
want to count the plumbing required to call DMain.  In that case, 
we can just call CMain directly:


  module test;

  extern(C) void main() { }

  Same Error:
  Error: cannot find source code for runtime library file 
'object.d'



Try with the -betterC switch:

  dmd -m64 -defaultlib= -debuglib= -conf= -betterC test.d

  Same Error:
  Error: cannot find source code for runtime library file 
'object.d'


Ok, fine, let's add an empty object.d

  //object.d
  module object;

  //test.d
  module test;

  extern(C) void main() { }

  dmd -m64 -defaultlib= -debuglib= -conf= -betterC object.d test.d

  Error: undefined identifier 'Error'
  Error: undefined identifier 'Error'

Ok, now I need something called 'Error'.  I don't see anywhere in 
my code where that's necessary, but it must be important because 
it told me twice.


  //object.d
  module object;

  class Throwable { }

  class Error : Throwable { }

  //test.d
  module test;

  extern(C) void main() { }

  dmd -m64 -defaultlib= -debuglib= -conf= -betterC object.d test.d

  object.d(3): Error: class object.Throwable missing or corrupt 
object.d


Ok, that's weird. I just added it.  Luckily I've done this 
before, so I know what to do.


  //object.d
  module object;

  class Object { }

  class Throwable { }

  class Error : Throwable
  {
  this(string x) { }
  }

  //test.d
  module test;

  extern(C) void main() { }

  dmd -m64 -defaultlib= -debuglib= -conf= -betterC object.d test.d

  Error: no constructor for Error
  Error: no constructor for Error

Hmm. Again with the double error messages.  Ok, let's add a 
constructor.


  //object.d
  module object;

  class Object { }

  class Throwable { }

  class Error : Throwable
  {
  this(string x) { }
  }

  //test.d
  module test;

  extern(C) void main() { }

  dmd -m64 -defaultlib= -debuglib= -conf= -betterC object.d test.d

  object.d(9): Error: undefined identifier 'string'

At least its only telling me once.  Easy fix.

  //object.d
  module object;

  alias immutable(char)[] string;

  class Object { }

  class Throwable { }

  class Error : Throwable
  {
  this(string x) { }
  }

  //test.d
  module test;

  extern(C) void main() { }

  dmd -m64 -defaultlib= -debuglib= -conf= -betterC object.d test.d

  Error: TypeInfo not found. object.d may be incorrectly 
installed or corrupt,

compile with -v switch

I could go on, but I'm not going to.  The final result to get a 
build looks like this, but you have to link with --gc-sections to 
avoid having to implement a buch of other stuff.


  module object;

  extern(C) void __dmd_personality_v0() { }

  extern(C) void _d_dso_registry() { }

  alias immutable(char)[] string;

  class Object { }

  class TypeInfo { }

  class TypeInfo_Class : TypeInfo
  {
  ubyte[136] ignore;
  }

  alias TypeInfo_Class ClassInfo;

  class Throwable { }

  class Error : Throwable
  {
  this(string x) { }
  }

  //test.d
  module test;

  extern(C) void main() { }

  dmd -m64 -defaultlib= -debuglib= -conf= -betterC object.d 
test.d -L=--gc-sections


If you add structs or classes to your code, you have to add even 
more druntime stuff.


The point I'm trying to make here is druntime is required just to 
get a build, even though your code doesn't need it.


Mike


Re: A betterC modular standard library?

2016-12-20 Thread Walter Bright via Digitalmars-d

On 12/20/2016 5:35 AM, Ilya Yaroshenko wrote:

On Tuesday, 20 December 2016 at 12:49:53 UTC, Walter Bright wrote:


I don't see the need to rewrite cpuid because its sole dependency on other
object files in Druntime is the static constructor, because the static
constructor can be adjusted to not rely on anything else in Druntime.

If there are other issues with cpuid, please elucidate.


1. https://issues.dlang.org/show_bug.cgi?id=16028
2. The code is very hard to maintain, fix, extend. This is a reason why
mir-cpuid was written from scratch without any copy-pasting.
3. Library is x86 oriented. Some OSs has interface to fetch unified CPU
information.
4. No TLB information provided.
5. Number of CPUs per cache is not know. ARM CPUs may have complex topology
(plus 3.)


Thank you, but I meant issues that would require it to be separate from 
druntime, not bug fixes or enhancements to account for ever more things being 
added to the CPUID instruction.


BTW, of course it is x86 oriented. Any use of cpuid must be. After all, what 
does "SSSE3.1" mean on ARM?


Re: A betterC modular standard library?

2016-12-20 Thread Ilya Yaroshenko via Digitalmars-d
On Tuesday, 20 December 2016 at 12:45:13 UTC, Guillaume Piolat 
wrote:

On Tuesday, 20 December 2016 at 12:33:27 UTC, John Colvin wrote:
* D used to have 2 standard libraries. That was not a happy 
time for the community. People are wary partly because of that.


To bring some cool to the discussion, let's remember that they 
also weren't compatible.
Of course it's a concern but this case is different and deserve 
full consideration. let's say the thread title was "A 
runtime-less modular library?"


Thanks! Good point


Re: A betterC modular standard library?

2016-12-20 Thread Ilya Yaroshenko via Digitalmars-d

On Tuesday, 20 December 2016 at 12:49:53 UTC, Walter Bright wrote:

I don't see the need to rewrite cpuid because its sole 
dependency on other object files in Druntime is the static 
constructor, because the static constructor can be adjusted to 
not rely on anything else in Druntime.


If there are other issues with cpuid, please elucidate.


1. https://issues.dlang.org/show_bug.cgi?id=16028
2. The code is very hard to maintain, fix, extend. This is a 
reason why mir-cpuid was written from scratch without any 
copy-pasting.
3. Library is x86 oriented. Some OSs has interface to fetch 
unified CPU information.

4. No TLB information provided.
5. Number of CPUs per cache is not know. ARM CPUs may have 
complex topology (plus 3.)


Re: A betterC modular standard library?

2016-12-20 Thread Ilya Yaroshenko via Digitalmars-d

On Tuesday, 20 December 2016 at 12:33:27 UTC, John Colvin wrote:

Mir uses deprecated native complex numbers because we can not 
spend days explaining why they are better then std.complex (in 
short: std.complex breaks compiler optimization logic, function 
inlining works, but it breaks optimization logic). I have already 
spend 4 hours for forum discussions and PRs about this issue 
without any result.


We are doing the best things we can do with current D. If 
something will be improved in future we will upgrade codebase.


P.S. a good example of the sort of statement that needs more 
explanation, at least for me:

The betterC
std.range and std.algorithm analogs would be released with new 
ndslice
implementation. Mir's algorithm would be faster then Phobos 
and will

generate less template bloat."
How? Why? Is ndslice actually capable of the same level of 
flexibility as std.algorithm and std.range? You're effectively 
saying "all that stuff that Andrei (and others) designed and 
wrote, I can do better" without really showing anyone why they 
should believe that.


The explanation of new idioms will appear after its code.

I did not say "better". I said "betterC" plus "faster".

Yes, i can say that for DCV library new idioms are much better 
then Phobos.


But new idioms have another issues, for example D language is not 
capable to infer their safety for many of them. Other issues is 
that there a lot of fast non CTFE-able types and their CTFE-able 
analogs.


One good things for safety and CTFE is allow multiple return 
value. In combination with `auto ref` it is _very_ powerful:



auto ref front()
{
  // Returns 2 values, each value is returned by reference if 
possible

  return (a.front, b.front);
}


This would be interesting to discuss and to have it in D in CTFE 
and @safe mode. But I am not good in DIPs.


This sorts of features is very desirable.



Re: A betterC modular standard library?

2016-12-20 Thread Walter Bright via Digitalmars-d

On 12/20/2016 4:30 AM, Ilya Yaroshenko wrote:

This is true for application but not for libraries. mir-glas depends on
mir-cpuid. If I want GLAS to be ported for a new target then I will release a
new CPUID version and will be able to use it the same day. With DRuntime (as
solid project), I need to wait up to 6 months before it will be released with
DMD and then with LDC. Workaround are possible, but they requires more efforts,
more time, more code.

Please do not force Mir to use DRuntime. We need a solid solution

I do not force anyone to drop DRuntime: Mir libraries can be used with
DRuntime/Phobos. We have full backward compatibility. DRuntime does not satisfy
portability requirements (which was repeated multiple times in this thread).
DRuntime (at least for now and future few years) is a big constraint for betterC
libs to replace existing C analogs.


Here's how anyone can override druntime's cpuid for their application without 
touching druntime:


dmd -c mycpuid
dmd myapp mycpuid.o

I.e. the way linkers work is if the symbol's definition exists, it is not pulled 
in from the library.


> Please do not force me to use DRuntime.

I don't propose forcing anyone to use Druntime. What I do propose is reducing 
interdependencies among the object files in Druntime. Druntime is not a 
monolithic chunk of code, it is a library of object files.


I don't see the need to rewrite cpuid because its sole dependency on other 
object files in Druntime is the static constructor, because the static 
constructor can be adjusted to not rely on anything else in Druntime.


If there are other issues with cpuid, please elucidate.


Re: A betterC modular standard library?

2016-12-20 Thread Guillaume Piolat via Digitalmars-d

On Tuesday, 20 December 2016 at 12:33:27 UTC, John Colvin wrote:
* D used to have 2 standard libraries. That was not a happy 
time for the community. People are wary partly because of that.


To bring some cool to the discussion, let's remember that they 
also weren't compatible.
Of course it's a concern but this case is different and deserve 
full consideration. let's say the thread title was "A 
runtime-less modular library?"


Re: A betterC modular standard library?

2016-12-20 Thread John Colvin via Digitalmars-d
On Tuesday, 20 December 2016 at 09:56:34 UTC, Ilya Yaroshenko 
wrote:

Of course you can bundle your own Phobos.
If you put a std.* module in your path your build will be 
working against that instead. If you don't mess around with 
sc.ini it will override the default for that compilation.


CPUID should be precompiled. So this does not work. If GLAS do 
not need anything but CPUID, what the reason to depend on 
DRuntime?


Please stop to force me to use DRuntime because you think it is 
better for Mir projects. This looks like DRuntime religion.


A betterC library can be used with DRuntime. So I do not force 
anyone to drop DRuntime. Please do not force me to use it.


GLAS already works without DRuntime and I am happy about that.


What does "you can bundle your own phobos" have to do with that? 
No one's forcing you to do anything.


You're suggesting something radical*, other people are suggesting 
that maybe there are good compromises where everybody wins, 
avoiding fragmentation. Then you say that "looks like DRuntime 
religion" and claim you're being forced to modify your code to 
add extra dependencies. It doesn't make sense.


Your technical arguments have good content, in my opinion 
everyone would benefit from you writing them up with sufficient 
context for people who don't know what you know and without 
hyperbole (none of that "D will fail if we don't do X" or "phobos 
is bloated and useless" stuff, it doesn't help communicate your 
points). Developing the case study / thought experiment of 
getting two libraries (blas and fft) in to a traditional linux 
distro would be a great central point.


See https://github.com/dlang/DIPs/pull/51 where Andrei describes 
not only how he thinks things should be, but also explains in 
detail why other approaches won't work. Now consider that what 
you're talking about is a lot more important and 
different/disruptive than a new import syntax, so it deserves at 
least comparably good description.


* D used to have 2 standard libraries. That was not a happy time 
for the community. People are wary partly because of that.


P.S. a good example of the sort of statement that needs more 
explanation, at least for me:

The betterC
std.range and std.algorithm analogs would be released with new 
ndslice
implementation. Mir's algorithm would be faster then Phobos and 
will

generate less template bloat."
How? Why? Is ndslice actually capable of the same level of 
flexibility as std.algorithm and std.range? You're effectively 
saying "all that stuff that Andrei (and others) designed and 
wrote, I can do better" without really showing anyone why they 
should believe that.


Re: A betterC modular standard library?

2016-12-20 Thread Ilya Yaroshenko via Digitalmars-d

On Tuesday, 20 December 2016 at 12:00:43 UTC, Walter Bright wrote:

On 12/18/2016 1:02 PM, Ilya Yaroshenko wrote:
I need to understand how it is can be done because I did not 
find a valid solution.


mir-cpuid has global symbols, they are accessed using 
extern(C) API. If multiple
libraries trying to initialize it than it will be initialized 
only once.


extern C API does not solve a problem with DRuntime because 
current DRuntime is
not a betterC library and it is huge. A huge betterC library 
is a problem too

because it is less portable.


cpuid can be written in a manner that it is self-contained. If 
so, it doesn't matter how large druntime is, as it won't have 
references to any of the rest of it and so the rest won't be 
pulled in by the linker.


This is true for application but not for libraries. mir-glas 
depends on mir-cpuid. If I want GLAS to be ported for a new 
target then I will release a new CPUID version and will be able 
to use it the same day. With DRuntime (as solid project), I need 
to wait up to 6 months before it will be released with DMD and 
then with LDC. Workaround are possible, but they requires more 
efforts, more time, more code.


Please do not force Mir to use DRuntime. We need a solid solution

I do not force anyone to drop DRuntime: Mir libraries can be used 
with DRuntime/Phobos. We have full backward compatibility. 
DRuntime does not satisfy portability requirements (which was 
repeated multiple times in this thread). DRuntime (at least for 
now and future few years) is a big constraint for betterC libs to 
replace existing C analogs. Please do not force me to use 
DRuntime.


Re: A betterC modular standard library?

2016-12-20 Thread Walter Bright via Digitalmars-d

On 12/18/2016 1:02 PM, Ilya Yaroshenko wrote:

I need to understand how it is can be done because I did not find a valid 
solution.

mir-cpuid has global symbols, they are accessed using extern(C) API. If multiple
libraries trying to initialize it than it will be initialized only once.

extern C API does not solve a problem with DRuntime because current DRuntime is
not a betterC library and it is huge. A huge betterC library is a problem too
because it is less portable.


cpuid can be written in a manner that it is self-contained. If so, it doesn't 
matter how large druntime is, as it won't have references to any of the rest of 
it and so the rest won't be pulled in by the linker.


https://github.com/dlang/druntime/blob/master/src/core/cpuid.d

Currently, core.cpuid has no imports. It does have a static constructor, though, 
which references the startup code in druntime. This can be changed so it uses 
the static construction support in the C runtime library. Then it will be 
independent of druntime.


Re: A betterC modular standard library?

2016-12-20 Thread Ilya Yaroshenko via Digitalmars-d
On Tuesday, 20 December 2016 at 09:38:58 UTC, rikki cattermole 
wrote:

On 20/12/2016 10:32 PM, Ilya Yaroshenko wrote:
On Tuesday, 20 December 2016 at 09:12:55 UTC, Timothee Cour 
wrote:

@ilya: regarding -betterC 's stated goal to bypass druntime:

besides static linking (which can be inefficient), why not 
use runtime

shared libraries to distribute mir (eg GLAS) to C clients ?

IIRC, from a single C++ program, I am able to dlopen & dlsym 
& run D
functions defined in shared libraries built with different 
compilers
(eg dmd and ldc) even if they use GC (and therefore dmd's and 
ldc's

druntime) ; pseudocode:

// main.cpp
```
void main(){
  dlopen, dlsym, and run "void fun1()" from libfun_dmd.so
  dlopen, dlsym, and run "void fun2()" from libfun_ldc.so
}
```

That would provide the needed isolation without suffering from
disadvantages of static libraries (eg bloat); compared to the 
betterC
alternative, you'd have access to all of druntime/phobos and 
not have

to reinvent the wheel


Because I do not have control on phobos, i can not distribute 
my copy of

phobos with required patches like normal cache size estimation
(core.cpuid is broken). Why not to fix core.cpuid?  Its source 
code is
not manageable comparing with mir-cpuid: it is faster to write 
a module
from scratch for this case. DUB package appears when I want 
it, LDC

release may appear after 6-10 months (depends on distribution).


Of course you can bundle your own Phobos.
If you put a std.* module in your path your build will be 
working against that instead. If you don't mess around with 
sc.ini it will override the default for that compilation.


CPUID should be precompiled. So this does not work. If GLAS do 
not need anything but CPUID, what the reason to depend on 
DRuntime?


Please stop to force me to use DRuntime because you think it is 
better for Mir projects. This looks like DRuntime religion.


A betterC library can be used with DRuntime. So I do not force 
anyone to drop DRuntime. Please do not force me to use it.


GLAS already works without DRuntime and I am happy about that.



Re: A betterC modular standard library?

2016-12-20 Thread rikki cattermole via Digitalmars-d

On 20/12/2016 10:32 PM, Ilya Yaroshenko wrote:

On Tuesday, 20 December 2016 at 09:12:55 UTC, Timothee Cour wrote:

@ilya: regarding -betterC 's stated goal to bypass druntime:

besides static linking (which can be inefficient), why not use runtime
shared libraries to distribute mir (eg GLAS) to C clients ?

IIRC, from a single C++ program, I am able to dlopen & dlsym & run D
functions defined in shared libraries built with different compilers
(eg dmd and ldc) even if they use GC (and therefore dmd's and ldc's
druntime) ; pseudocode:

// main.cpp
```
void main(){
  dlopen, dlsym, and run "void fun1()" from libfun_dmd.so
  dlopen, dlsym, and run "void fun2()" from libfun_ldc.so
}
```

That would provide the needed isolation without suffering from
disadvantages of static libraries (eg bloat); compared to the betterC
alternative, you'd have access to all of druntime/phobos and not have
to reinvent the wheel


Because I do not have control on phobos, i can not distribute my copy of
phobos with required patches like normal cache size estimation
(core.cpuid is broken). Why not to fix core.cpuid?  Its source code is
not manageable comparing with mir-cpuid: it is faster to write a module
from scratch for this case. DUB package appears when I want it, LDC
release may appear after 6-10 months (depends on distribution).


Of course you can bundle your own Phobos.
If you put a std.* module in your path your build will be working 
against that instead. If you don't mess around with sc.ini it will 
override the default for that compilation.





Re: A betterC modular standard library?

2016-12-20 Thread Ilya Yaroshenko via Digitalmars-d

On Tuesday, 20 December 2016 at 09:12:55 UTC, Timothee Cour wrote:

@ilya: regarding -betterC 's stated goal to bypass druntime:

besides static linking (which can be inefficient), why not use 
runtime

shared libraries to distribute mir (eg GLAS) to C clients ?

IIRC, from a single C++ program, I am able to dlopen & dlsym & 
run D functions defined in shared libraries built with 
different compilers (eg dmd and ldc) even if they use GC (and 
therefore dmd's and ldc's druntime) ; pseudocode:


// main.cpp
```
void main(){
  dlopen, dlsym, and run "void fun1()" from libfun_dmd.so
  dlopen, dlsym, and run "void fun2()" from libfun_ldc.so
}
```

That would provide the needed isolation without suffering from 
disadvantages of static libraries (eg bloat); compared to the 
betterC alternative, you'd have access to all of 
druntime/phobos and not have to reinvent the wheel


Because I do not have control on phobos, i can not distribute my 
copy of phobos with required patches like normal cache size 
estimation (core.cpuid is broken). Why not to fix core.cpuid?  
Its source code is not manageable comparing with mir-cpuid: it is 
faster to write a module from scratch for this case. DUB package 
appears when I want it, LDC release may appear after 6-10 months 
(depends on distribution).




Re: A betterC modular standard library?

2016-12-20 Thread Timothee Cour via Digitalmars-d
@ilya: regarding -betterC 's stated goal to bypass druntime:

besides static linking (which can be inefficient), why not use runtime
shared libraries to distribute mir (eg GLAS) to C clients ?

IIRC, from a single C++ program, I am able to dlopen & dlsym & run D
functions defined in shared libraries built with different compilers (eg
dmd and ldc) even if they use GC (and therefore dmd's and ldc's druntime) ;
pseudocode:

// main.cpp
```
void main(){
  dlopen, dlsym, and run "void fun1()" from libfun_dmd.so
  dlopen, dlsym, and run "void fun2()" from libfun_ldc.so
}
```

That would provide the needed isolation without suffering from
disadvantages of static libraries (eg bloat); compared to the betterC
alternative, you'd have access to all of druntime/phobos and not have to
reinvent the wheel


On Tue, Dec 20, 2016 at 1:06 AM, Paolo Invernizzi via Digitalmars-d <
digitalmars-d@puremagic.com> wrote:

> On Tuesday, 20 December 2016 at 08:15:35 UTC, Dicebot wrote:
>
>> On 12/18/2016 11:26 AM, Ilya Yaroshenko wrote:>
>>
>> This actually may be even possible to do within a linear deprecation
>> process. On the other hand I doubt such drastic change to structure is
>> possible to be sold to community.
>>
>
> Consider that plan sold to me...
>
> ---
> Paolo
>
>


Re: A betterC modular standard library?

2016-12-20 Thread Paolo Invernizzi via Digitalmars-d

On Tuesday, 20 December 2016 at 08:15:35 UTC, Dicebot wrote:

On 12/18/2016 11:26 AM, Ilya Yaroshenko wrote:>

This actually may be even possible to do within a linear 
deprecation process. On the other hand I doubt such drastic 
change to structure is possible to be sold to community.


Consider that plan sold to me...

---
Paolo



Re: A betterC modular standard library?

2016-12-20 Thread Dicebot via Digitalmars-d
On 12/18/2016 11:26 AM, Ilya Yaroshenko wrote:
> Hi,
> 
> Who is interested in betterC _modular_* standard library?
> I am planing to make libmir org a community for it.
> Thought and concerns?

I also consider Phobos a lost cause only suitable for scripting
purposes. However what you try to do won't be acceptable for me either
(see below).

> We already have better `cpuid` and better `random` packages. The betterC
> std.range and std.algorithm analogs would be released with new ndslice
> implementation. Mir's algorithm would be faster then Phobos and will
> generate less template bloat. Then lightweight threads, multithread
> GLAS, matrix inversion. Fastest I/O and http2 ...

I consider only algorithms and interfaces suitable for standard library.
GLAS, I/O, HTTP - this is exactly all the stuff I want to see removed
into separate dub packages and maintained as such.

> (they all are betterC modular libraries)

.. and this "betterC" weirdness you use with your own meaning is
definitely something I don't want to _ever_ see as standard library
approach. It is similar to `malloc` vs `GC` debate for Phobos, because
correct answer tends to be "neither", with rationale that any library
utility that has to care about the difference is hardly generic enough
for unconditional distribution.



My own dream path for fixing Phobos may look like this:

1) Start with separating environment-agnostic core (algorithms and
API's) from utilities
2) Rewrite all higher level utilities in a way that they don't have any
inter-dependencies and only allowed to use algorithm/API modules
3) Move higher level stuff into own dub packages with a special "Phobos"
category
4) When compiler is released, provide snapshot of latest up to date
packages in that category as part of the distribution for those who
don't want to bother with fetching manually

This actually may be even possible to do within a linear deprecation
process. On the other hand I doubt such drastic change to structure is
possible to be sold to community.



signature.asc
Description: OpenPGP digital signature


Re: A betterC modular standard library?

2016-12-19 Thread Ilya Yaroshenko via Digitalmars-d

On Monday, 19 December 2016 at 21:00:57 UTC, lurker wrote:
On Monday, 19 December 2016 at 05:59:07 UTC, Ilya Yaroshenko 
wrote:

[...]



Thats why I asked for moderation in forums.



+1, and ability to edit own posts.


Re: A betterC modular standard library?

2016-12-19 Thread lurker via Digitalmars-d
On Monday, 19 December 2016 at 05:59:07 UTC, Ilya Yaroshenko 
wrote:
On Monday, 19 December 2016 at 01:52:21 UTC, Andrei 
Alexandrescu wrote:

On 12/18/16 7:31 PM, Seb wrote:

tl:dr: Phobos is very bloated.


How does the D standard library compare in size with some 
other languages (C++, Rust, Go, Java, Python)?


C++ and Rust are the best from this list in terms of low level 
programming. They have smaller libraries comparing with D.


Go, Java, Python --- If I need a language like them I will you 
them, not D.


Go - awesome GC, commercially supported, easy to write your own 
libs

Java - awesome GC, commercially supported, Scala(!!!)
Python - awesome scripting, commercially supported

D - ??? awesome what ? What do we have better then Scala?

Performance? NO.
BRandomAccessRanges makes our performance less then C/C++.

Clean and clever API in standard library? NO.
We are placing our Range API whenever possible including RNGs. 
We have not a language standard for I/O. Half of Phobos is 
outdated to language standards.


Best generics? NO.
Scala is more powerful.

--
What do we have awesome? Where D is the best?
--
D is awesome for low-level system libraries in betterC mode.
(and small text processing routines)
This will make D great in other fields.
--


Let's do something about it!


I've said this several times, and it bears repeating. We want 
to make D entirely pay-as-you go, and offer full support to 
anyone working on it. The best "something" anyone can do about 
it is do good engineering work in the D mainline.


National D Foundation Party.

That will automatically benefit all users of the D language, 
instead of balkanizing the community.


...

Granted, the lure of the clean slate is strong, and more care 
is needed to mind backwards compatibility, but that's where 
the real impact is.



Andrei


How Mir projects breaks backward compatibility ?

How proposed direction breaks backward compatibility?

Lets use other prefix for the new libraries. Mir uses `mir.` 
And allow Phobos to be a sandbox for C++ as it is now.


Ilya



Thats why I asked for moderation in forums.



Re: A betterC modular standard library?

2016-12-19 Thread Guillaume Piolat via Digitalmars-d
On Monday, 19 December 2016 at 20:10:02 UTC, Ilya Yaroshenko 
wrote:
On Monday, 19 December 2016 at 19:59:01 UTC, Guillaume Piolat 
wrote:
On Monday, 19 December 2016 at 19:54:54 UTC, Ilya Yaroshenko 
wrote:


To be more precise: I do not build library exclusively for D. 
They should be easily interacted with other languages.


Non sequitur, you can enable the runtime/attah threads withing 
callbacks.


No, DRuntime based numerical libraries will never replace C 
analogs.


Going to bed. Please don't take it as attacks, I'm juste trying 
to help find the best possible solution, with the least amount of 
pain.


Re: A betterC modular standard library?

2016-12-19 Thread Ilya Yaroshenko via Digitalmars-d
On Monday, 19 December 2016 at 20:24:21 UTC, Guillaume Piolat 
wrote:
On Monday, 19 December 2016 at 20:05:20 UTC, Ilya Yaroshenko 
wrote:

[...]


If Mir libs are static libs this can indeed be a problem.
If someone has made another D library like this, it will either 
have to be its own dynlib or come with a runtime with the same 
API. You can manage to use very little of druntime such as 
"emplace"/"destroy" and that's about it when it comes to API.


[...]


No, Mir is not a hardware manufacturer which forces its users to 
do something.


Note that with MIR being several dynlibs everyone of Mir 
library can come with any runtime (and then use it or not) 
without interaction of these runtimes.


[...]


Looks like bad design. Existing C libs would be better then.


Re: A betterC modular standard library?

2016-12-19 Thread Iain Buclaw via Digitalmars-d
On 19 December 2016 at 21:29, Ilya Yaroshenko via Digitalmars-d
 wrote:
> On Monday, 19 December 2016 at 20:22:29 UTC, Iain Buclaw wrote:
>>
>> On 19 December 2016 at 20:42, Ilya Yaroshenko via Digitalmars-d
>>  wrote:
>>>
>>> On Monday, 19 December 2016 at 19:31:24 UTC, Guillaume Piolat wrote:


 On Sunday, 18 December 2016 at 09:26:09 UTC, Ilya Yaroshenko wrote:
>
>
> [...]



 If I understand correctly, the only reaon not to disable the runtime
 instead of going betterC (what I call runtime-free vs runtime-less) is
 because Phobos and druntime aren't ported to every platform you need, but
 the libC is.
>>>
>>>
>>>
>>> No,
>>> 1. DRuntime has not fixed API
>>> 2. It has not backward binary compatibility
>>> 3. It has not cross-compiler binary compatibility.
>>>
>>> So, it can not be used.
>>
>>
>> You're confusing druntime with phobos.  I haven't changed any bindings to
>> druntime in years...
>
>
> If someone build a library with GCC and another one with Clang they will
> work together. What about LDC and GDC?

Clang developers decided that they should be ABI compatible with GCC.
I'm not sure if the LDC developers are doing the same thing. :-)


Re: A betterC modular standard library?

2016-12-19 Thread Ilya Yaroshenko via Digitalmars-d

On Monday, 19 December 2016 at 20:22:29 UTC, Iain Buclaw wrote:
On 19 December 2016 at 20:42, Ilya Yaroshenko via Digitalmars-d 
 wrote:
On Monday, 19 December 2016 at 19:31:24 UTC, Guillaume Piolat 
wrote:


On Sunday, 18 December 2016 at 09:26:09 UTC, Ilya Yaroshenko 
wrote:


[...]



If I understand correctly, the only reaon not to disable the 
runtime instead of going betterC (what I call runtime-free vs 
runtime-less) is because Phobos and druntime aren't ported to 
every platform you need, but the libC is.



No,
1. DRuntime has not fixed API
2. It has not backward binary compatibility
3. It has not cross-compiler binary compatibility.

So, it can not be used.


You're confusing druntime with phobos.  I haven't changed any 
bindings to druntime in years...


If someone build a library with GCC and another one with Clang 
they will work together. What about LDC and GDC?


Re: A betterC modular standard library?

2016-12-19 Thread Iain Buclaw via Digitalmars-d
On 19 December 2016 at 20:42, Ilya Yaroshenko via Digitalmars-d
 wrote:
> On Monday, 19 December 2016 at 19:31:24 UTC, Guillaume Piolat wrote:
>>
>> On Sunday, 18 December 2016 at 09:26:09 UTC, Ilya Yaroshenko wrote:
>>>
>>> [...]
>>
>>
>> If I understand correctly, the only reaon not to disable the runtime
>> instead of going betterC (what I call runtime-free vs runtime-less) is
>> because Phobos and druntime aren't ported to every platform you need, but
>> the libC is.
>
>
> No,
> 1. DRuntime has not fixed API
> 2. It has not backward binary compatibility
> 3. It has not cross-compiler binary compatibility.
>
> So, it can not be used.

You're confusing druntime with phobos.  I haven't changed any bindings
to druntime in years...


Re: A betterC modular standard library?

2016-12-19 Thread Guillaume Piolat via Digitalmars-d
On Monday, 19 December 2016 at 20:05:20 UTC, Ilya Yaroshenko 
wrote:
What about other non Mir numerical library written in Dlang, 
say for FFT?


How they should be linked together.


If Mir libs are static libs this can indeed be a problem.
If someone has made another D library like this, it will either 
have to be its own dynlib or come with a runtime with the same 
API. You can manage to use very little of druntime such as 
"emplace"/"destroy" and that's about it when it comes to API.



I'll concede that my solution will not get you packaged on 
official package lists for Debian/Ubuntu, because shared 
libraries will get preferred there.


But I just checked and libcublas is just put in the non-free 
packages.

https://packages.debian.org/jessie/libcublas6.0
If it's good enough for CUDA then maybe it's good enough for MIR?

Note that with MIR being several dynlibs everyone of Mir library 
can come with any runtime (and then use it or not) without 
interaction of these runtimes.


When you use a CUDA dynamic library you don't know if it has a C 
runtime inside (or a X runtime, whatever).



Do you propose to constraint Dlang library distribution in 
binary form?


No. Not sure how that question ensues from the premises.







Re: A betterC modular standard library?

2016-12-19 Thread Ilya Yaroshenko via Digitalmars-d
On Monday, 19 December 2016 at 19:59:01 UTC, Guillaume Piolat 
wrote:
On Monday, 19 December 2016 at 19:54:54 UTC, Ilya Yaroshenko 
wrote:


To be more precise: I do not build library exclusively for D. 
They should be easily interacted with other languages.


Non sequitur, you can enable the runtime/attah threads withing 
callbacks.


No, DRuntime based numerical libraries will never replace C 
analogs.


Re: A betterC modular standard library?

2016-12-19 Thread Ilya Yaroshenko via Digitalmars-d
On Monday, 19 December 2016 at 19:57:45 UTC, Guillaume Piolat 
wrote:
On Monday, 19 December 2016 at 19:52:43 UTC, Ilya Yaroshenko 
wrote:

How this can work for libraries? Please read this thread.
This is not realistic. This a huge constraint for D libraries 
to be modular, replaceable and distributed in binary form.


They can be at the MIR level.

How this can work: Mir libraries are distributed as static and 
dynamic libs, whatever.
Each one of these links statically with the same druntime and 
libphobos. When the user replace one MIR library it also 
replaces the others.


I don't see what isn't realistic here.

See also: CUDA, Intel Performance Primitives...
People update CUDA in bunch, they don't use cuDNN from release 
6.15 with cuRuntime from release 7.8


Mir CPUID was created for Mir GLAS, which is BLAS implementation.

How Mir GLAS be a default BLAS implementation in linux 
distributives if it is depends on DRuntime?


What about other non Mir numerical library written in Dlang, say 
for FFT?


How they should be linked together.

This example is already in this thread multiple times.

Do you propose to constraint Dlang library distribution in binary 
form?


I do not write D libraries for Dlang. I wrote them for people, 
which uses different languages. Most of them do not know about D.


Re: A betterC modular standard library?

2016-12-19 Thread Guillaume Piolat via Digitalmars-d
On Monday, 19 December 2016 at 19:54:54 UTC, Ilya Yaroshenko 
wrote:


To be more precise: I do not build library exclusively for D. 
They should be easily interacted with other languages.


Non sequitur, you can enable the runtime/attah threads withing 
callbacks.


Re: A betterC modular standard library?

2016-12-19 Thread Guillaume Piolat via Digitalmars-d
On Monday, 19 December 2016 at 19:52:43 UTC, Ilya Yaroshenko 
wrote:

How this can work for libraries? Please read this thread.
This is not realistic. This a huge constraint for D libraries 
to be modular, replaceable and distributed in binary form.


They can be at the MIR level.

How this can work: Mir libraries are distributed as static and 
dynamic libs, whatever.
Each one of these links statically with the same druntime and 
libphobos. When the user replace one MIR library it also replaces 
the others.


I don't see what isn't realistic here.

See also: CUDA, Intel Performance Primitives...
People update CUDA in bunch, they don't use cuDNN from release 
6.15 with cuRuntime from release 7.8






Re: A betterC modular standard library?

2016-12-19 Thread Ilya Yaroshenko via Digitalmars-d
On Monday, 19 December 2016 at 19:52:43 UTC, Ilya Yaroshenko 
wrote:
On Monday, 19 December 2016 at 19:48:39 UTC, Guillaume Piolat 
wrote:
On Monday, 19 December 2016 at 19:42:42 UTC, Ilya Yaroshenko 
wrote:

No,
1. DRuntime has not fixed API


Can't you static link with it? If you static link with it, 
your client don't deal with DRuntime.




2. It has not backward binary compatibility


Ditto. If you static link with it, your client don't deal with 
binary compatibility.




3. It has not cross-compiler binary compatibility.


Mandate that every D code in the project is built with the 
same compiler, which is realistic.


How this can work for libraries? Please read this thread.
This is not realistic. This a huge constraint for D libraries 
to be modular, replaceable and distributed in binary form.


To be more precise: I do not build library exclusively for D. 
They should be easily interacted with other languages.


Re: A betterC modular standard library?

2016-12-19 Thread Ilya Yaroshenko via Digitalmars-d
On Monday, 19 December 2016 at 19:48:39 UTC, Guillaume Piolat 
wrote:
On Monday, 19 December 2016 at 19:42:42 UTC, Ilya Yaroshenko 
wrote:

No,
1. DRuntime has not fixed API


Can't you static link with it? If you static link with it, your 
client don't deal with DRuntime.




2. It has not backward binary compatibility


Ditto. If you static link with it, your client don't deal with 
binary compatibility.




3. It has not cross-compiler binary compatibility.


Mandate that every D code in the project is built with the same 
compiler, which is realistic.


How this can work for libraries? Please read this thread.
This is not realistic. This a huge constraint for D libraries to 
be modular, replaceable and distributed in binary form.




Re: A betterC modular standard library?

2016-12-19 Thread Guillaume Piolat via Digitalmars-d
On Monday, 19 December 2016 at 19:42:42 UTC, Ilya Yaroshenko 
wrote:

No,
1. DRuntime has not fixed API


Can't you static link with it? If you static link with it, your 
client don't deal with DRuntime.




2. It has not backward binary compatibility


Ditto. If you static link with it, your client don't deal with 
binary compatibility.




3. It has not cross-compiler binary compatibility.


Mandate that every D code in the project is built with the same 
compiler, which is realistic.





Re: A betterC modular standard library?

2016-12-19 Thread Ilya Yaroshenko via Digitalmars-d
On Monday, 19 December 2016 at 19:31:24 UTC, Guillaume Piolat 
wrote:
On Sunday, 18 December 2016 at 09:26:09 UTC, Ilya Yaroshenko 
wrote:

[...]


If I understand correctly, the only reaon not to disable the 
runtime instead of going betterC (what I call runtime-free vs 
runtime-less) is because Phobos and druntime aren't ported to 
every platform you need, but the libC is.


No,
1. DRuntime has not fixed API
2. It has not backward binary compatibility
3. It has not cross-compiler binary compatibility.

So, it can not be used.


Re: A betterC modular standard library?

2016-12-19 Thread Guillaume Piolat via Digitalmars-d
On Sunday, 18 December 2016 at 09:26:09 UTC, Ilya Yaroshenko 
wrote:

Hi,

Who is interested in betterC _modular_* standard library?
I am planing to make libmir org a community for it.
Thought and concerns?

We already have better `cpuid` and better `random` packages. 
The betterC std.range and std.algorithm analogs would be 
released with new ndslice implementation. Mir's algorithm would 
be faster then Phobos and will generate less template bloat. 
Then lightweight threads, multithread GLAS, matrix inversion. 
Fastest I/O and http2 ...


(they all are betterC modular libraries)

Andrei and probably 90% of existing D users don't want Phobos 
to move this direction.


In other hand I need a commercial attractive D infrastructure 
for large and heavy system projects. There is no commercial 
perspective for me to contribute to Phobos because:


If I understand correctly, the only reaon not to disable the 
runtime instead of going betterC (what I call runtime-free vs 
runtime-less) is because Phobos and druntime aren't ported to 
every platform you need, but the libC is.


That it calling for the question: "Isn't porting Phobos and 
druntime to those new platforms your clients need less work?"


Maybe they only need ARM and x86-64, the effort should be minimal 
then? And with the runtime disabled, things like the GC just need 
to link, not actually work :)

Tha makes porting a bit simpler.

Things like std.algorithm, std.range, traits etc... took many 
man-years to do and can work as is.

core.cpuid can work with little modifications.

I'm not saying you can use a lot of Phobos and druntime without 
using the runtime, because currently you can't.
But think about it: when not linking the runtime, there isn't 
even a way to call "emplace" or "destroy", dynamic casts won't 
work, etc.


In the future more and more things will work without the runtime, 
exceptions, class creation and then you'll don't have to maintain 
a stdlib.


Please reconsider reconsidering.


Re: A betterC modular standard library?

2016-12-19 Thread Ilya Yaroshenko via Digitalmars-d

On Monday, 19 December 2016 at 18:04:14 UTC, safety0ff wrote:
On Sunday, 18 December 2016 at 18:02:58 UTC, Ilya Yaroshenko 
wrote:


Thank you for the answer (it is hard to understand me because 
English and other reasons),


Ilya


It was difficult to understand your vision until this post, now 
I think I grasp it.


Let me try to summarize what I've understood:
[...]


Yes, thank you for the clarification!


P.S.: I think Ilya writes "evaluates" where he means "evolves"


Oh, yes! I mean "evolves"!


Re: A betterC modular standard library?

2016-12-19 Thread safety0ff via Digitalmars-d
On Sunday, 18 December 2016 at 18:02:58 UTC, Ilya Yaroshenko 
wrote:


Thank you for the answer (it is hard to understand me because 
English and other reasons),


Ilya


It was difficult to understand your vision until this post, now I 
think I grasp it.


Let me try to summarize what I've understood:

D as it stands, is not suitable for writing low level libraries 
as well as large scale software development because of compiler 
dependence of compiled code.


Examples:
If you have two software teams, and team A's software dependent 
on compiler X (e.g requires newer feature, requires 
bug/regression fix, etc) and team B's software depends on 
compiler Y to meet performance requirements, they get stuck.


Also you want to create a low level library than can be easily 
distributed and linked from other languages (e.g. GLAS) extern 
(C) is the only viable option, but that can still lock in the D 
compiler used if you depend on phobos/druntime.



So the proposal is to make binary compatibility possible in the 
near future by implementing "betterC" which provides a bare-bones 
language and removes the greatest sources of incompatibilities.


Once this is done a community can form around it and create 
completely modular libraries. These can be used by all D and 
non-D users alike without compatibility problems.



Since this is all predicated on "betterC", which isn't 
implemented. I think it is imperative to create a full 
specification.


I look forward to seeing where this initiative goes.

P.S.: I think Ilya writes "evaluates" where he means "evolves"


Re: A betterC modular standard library?

2016-12-19 Thread yawniek via Digitalmars-d
On Monday, 19 December 2016 at 05:59:07 UTC, Ilya Yaroshenko 
wrote:

D - ??? awesome what ? What do we have better then Scala?

Performance? NO.
BRandomAccessRanges makes our performance less then C/C++.

Clean and clever API in standard library? NO.
We are placing our Range API whenever possible including RNGs. 
We have not a language standard for I/O. Half of Phobos is 
outdated to language standards.


Best generics? NO.
Scala is more powerful.

--
What do we have awesome? Where D is the best?
--
D is awesome for low-level system libraries in betterC mode.
(and small text processing routines)
This will make D great in other fields.


while i mostly use D as a awesome high level language (with 
mediocre libs, and an inconsistent stdlib) i also think that D 
has an edge when it comes to building fast system level tools 
that used to be written in c: microservices, datatabases etc.
however for that, as ilya mentioned it needs to be super user 
friendly to compile down a single static object/binary with no 
dependencies and a clear line on when phobos can/should/must be 
used and when not (e.g. create a view on phobos documentation 
that only shows things that work without druntime. add some 
official docs on how to use it)


people turn to C to extend their language of choice because they 
need speed in a very narrow area. if that becomes as easy (or 
even easier, since there is dub!) as with C it would be a huge 
win for D, because people would start to extend their stuff with 
D. No other language is in a such good position as D, from what i 
see not that much needs to be done to make that work.

i see a huge opportunity there





Re: A betterC modular standard library?

2016-12-18 Thread Andrey via Digitalmars-d
On Sunday, 18 December 2016 at 19:28:24 UTC, Andrei Alexandrescu 
wrote:

On 12/18/2016 01:49 PM, Ilya Yaroshenko wrote:
On Sunday, 18 December 2016 at 17:41:31 UTC, Walter Bright 
wrote:

On 12/18/2016 1:26 AM, Ilya Yaroshenko wrote:

We already have better `cpuid` and better `random` packages.


Great! Please PR them for Phobos.


cpuid is used in Mir GLAS and it should be a betterC library.


It's this kind of imaginary dialog that I don't quite grok:

Ilya Yaroshenko: "I can't use druntime's cpuid so I defined my 
own."


Andrei Alexandrescu: "OK, so what's different?"

IY: "Mine has a few engineering improvements."

AA: "Cool, why don't you merge them into core.cpuid?"

IY: "Well mine doesn't have a shared static constructor, so it 
doesn't need a runtime to automatically call that library 
initialization function. User need to explicitly call an init 
function before using it."


AA: "I understand. Great, so how about this - we add your code 
to core.cpuid_v2 to druntime. Then we make all in core.cpuid to 
forward to it so there's no duplication. It all works out!"


IY: "No, I don't want to do that. It's still in druntime and I 
don't want druntime. I want betterC."


AA: "But it will be compilable with betterC and we can add 
unittests to make that happen. YOU HAVE MY SUPPORT. Let's do 
it."


IY: "No, I want to change it often. The deployment schedule of 
druntime is too slow."


AA: "How often do you need to change it? Is it that unstable?"

IY: "Um, not too often."

AA: "Then what is the matter? Are you worried about the IP of 
the engineering improvements you are making? Are you licensing 
this differently?"


IY: "No, it's for the most part similar to core.cpuid and the 
license will be also Boost."


AA: "Then what is the matter? Do you want me to wait until you 
release a stable mir.cpuid and copy it over with credit, per 
the Boost license, to core.cpuid_v2?"


IY: "..."

It's this kind of stuff I need to have a better understanding 
of. Some technical arguments are meaningful, some others point 
to problems with obvious solutions that are somehow shunned, 
and yet others are like a hidden Markov model - I see the 
effects, but I don't see the causes.



Andrei


I think, the reason of misunderstanding is different skills in a 
subject. And even different subjects. Some experts are compilers 
developers, some are library developers. They all in different 
subject. In c++ community there is a group of compiler developers 
experts who desides, how to implement this or that feature, and 
the ability of implementation. They know how to develop 
compilers. It doesn't mean they are more clever than library 
developers, just different skills. And when Andrei asking to 
explain the reason of some proposal, he waits arguments in 
_terminology_ of comliler developer. _terminology_ understanding 
(in wide) means _experience_ (compiler developing in this case). 
If no experience in the subject, there is a way to get it, and 
the other way to ask a recommendation of experienced colleague.


Re: A betterC modular standard library?

2016-12-18 Thread Mike via Digitalmars-d
On Monday, 19 December 2016 at 05:22:10 UTC, Ilya Yaroshenko 
wrote:
What exactly do you mean by "betterC"?  As far as I know, the 
`-betterC` switch just removes ModuleInfo generation.


Because -betterC was not implemented yet.


I'm curious to know what a complete implementation entails.

Do you actually mean, a standard library that has fewer 
dependencies on druntime features?


I mean this too


I've been wanting to decouple from druntime since I started 
learning D in late 2013, though probably for different reasons 
than yours.  There have been two recent posts [1] [2] wanting to 
do the same thing, and now your post envisioning a more modular 
druntime-less phobos.


[1] - 
https://forum.dlang.org/post/drqdrobcoaanenocy...@forum.dlang.org
[2] - 
https://forum.dlang.org/post/tmofjecvnqdthvete...@forum.dlang.org


It's encouraging to see this trend.  I am interested in seeing D 
become more modular and pay-as-you-go, though I'd prefer to see 
changes in the compiler and druntime (e.g. [3] [4] [5]) before 
phobos.


[3] - http://forum.dlang.org/post/mr71kt$2rgi$1...@digitalmars.com
[4] - 
http://forum.dlang.org/post/eiwalbqlbkipdrmsr...@forum.dlang.org

[5] - https://issues.dlang.org/show_bug.cgi?id=11666

Mike


Re: A betterC modular standard library?

2016-12-18 Thread Ilya Yaroshenko via Digitalmars-d
On Monday, 19 December 2016 at 01:52:21 UTC, Andrei Alexandrescu 
wrote:

On 12/18/16 7:31 PM, Seb wrote:

tl:dr: Phobos is very bloated.


How does the D standard library compare in size with some other 
languages (C++, Rust, Go, Java, Python)?


C++ and Rust are the best from this list in terms of low level 
programming. They have smaller libraries comparing with D.


Go, Java, Python --- If I need a language like them I will you 
them, not D.


Go - awesome GC, commercially supported, easy to write your own 
libs

Java - awesome GC, commercially supported, Scala(!!!)
Python - awesome scripting, commercially supported

D - ??? awesome what ? What do we have better then Scala?

Performance? NO.
BRandomAccessRanges makes our performance less then C/C++.

Clean and clever API in standard library? NO.
We are placing our Range API whenever possible including RNGs. We 
have not a language standard for I/O. Half of Phobos is outdated 
to language standards.


Best generics? NO.
Scala is more powerful.

--
What do we have awesome? Where D is the best?
--
D is awesome for low-level system libraries in betterC mode.
(and small text processing routines)
This will make D great in other fields.
--


Let's do something about it!


I've said this several times, and it bears repeating. We want 
to make D entirely pay-as-you go, and offer full support to 
anyone working on it. The best "something" anyone can do about 
it is do good engineering work in the D mainline.


National D Foundation Party.

That will automatically benefit all users of the D language, 
instead of balkanizing the community.


...

Granted, the lure of the clean slate is strong, and more care 
is needed to mind backwards compatibility, but that's where the 
real impact is.



Andrei


How Mir projects breaks backward compatibility ?

How proposed direction breaks backward compatibility?

Lets use other prefix for the new libraries. Mir uses `mir.` And 
allow Phobos to be a sandbox for C++ as it is now.


Ilya



Re: A betterC modular standard library?

2016-12-18 Thread Ilya Yaroshenko via Digitalmars-d

On Monday, 19 December 2016 at 01:33:42 UTC, Mike wrote:
On Sunday, 18 December 2016 at 09:26:09 UTC, Ilya Yaroshenko 
wrote:



Who is interested in betterC _modular_* standard library?


What exactly do you mean by "betterC"?  As far as I know, the 
`-betterC` switch just removes ModuleInfo generation.


Because -betterC was not implemented yet.

Do you actually mean, a standard library that has fewer 
dependencies on druntime features?


I mean this too

Ilya



Re: A betterC modular standard library?

2016-12-18 Thread Andrei Alexandrescu via Digitalmars-d

On 12/18/16 7:31 PM, Seb wrote:

tl:dr: Phobos is very bloated.


How does the D standard library compare in size with some other 
languages (C++, Rust, Go, Java, Python)?



Let's do something about it!


I've said this several times, and it bears repeating. We want to make D 
entirely pay-as-you go, and offer full support to anyone working on it. 
The best "something" anyone can do about it is do good engineering work 
in the D mainline. That will automatically benefit all users of the D 
language, instead of balkanizing the community. Granted, the lure of the 
clean slate is strong, and more care is needed to mind backwards 
compatibility, but that's where the real impact is.



Andrei



Re: A betterC modular standard library?

2016-12-18 Thread Mike via Digitalmars-d
On Sunday, 18 December 2016 at 09:26:09 UTC, Ilya Yaroshenko 
wrote:



Who is interested in betterC _modular_* standard library?


What exactly do you mean by "betterC"?  As far as I know, the 
`-betterC` switch just removes ModuleInfo generation.


Do you actually mean, a standard library that has fewer 
dependencies on druntime features?


Mike




Re: A betterC modular standard library?

2016-12-18 Thread Seb via Digitalmars-d
On Sunday, 18 December 2016 at 18:49:33 UTC, Ilya Yaroshenko 
wrote:
1. Modularity: D should provide a very slim library (like 
std.traits and may be default GC with old core, which can be 
turned off or replaced). Other parts or Phobos/Druntime should 
be split into parts and the parts should have their own 
repositories. They should be dub packages with D Foundation 
review / control.


This would be awesome to have!!

tl:dr: Phobos is very bloated. Let's do something about it!

I absolutely agree that there's a lot of stuff that doesn't 
belong into Phobos.
Also many parts are outdated or have ugly APIs, but they can only 
be replaced if a superior community solution has evolved.

So why not "dubify" the optional parts of Phobos?

Solved Problem: Versioning
--

This also solves the versioning problem Andrei discussed last 
month nicely. For example when std.allocator was merged into 
Phobos a lot of builds broke because packages did can their 
dependency from dub to std.x.allocator, but many people were 
still compiling with an older compiler.


Solved Problem: High quality community libraries


Moreover, at the last DConf there were many voices that D is 
lacking good community libraries (aka the gold libraries). So the 
Phobos collection could be one.


Btw one shouldn't forgot that people have already written a lot 
of replacements of Phobos modules and imho in most cases it made 
sense!


A quick list


(this list is incomplete and intended as an example to show that 
for most parts of Phobos better solutions co-exists)


etc.c.sqlite (aka std.database)

-> https://github.com/buggins/ddbc
(and: https://wiki.dlang.org/Database_Libraries)

etc.c.zlib / std.zip

-> https://github.com/rcythr/archive

std.base64

-> https://code.dlang.org/packages/base-d

std.bigint

-> 
https://github.com/andersonpd/eris/blob/master/integer/extended.d


std.csv

-> https://github.com/eBay/tsv-utils-dlang

std.container

-> https://github.com/economicmodeling/containers (uses 
allocators!)


std.complex

I am pretty sure this is on Ilya's list as well ;-)

std.encoding:

-> https://github.com/e10s/d-base32

std.getopt

-> https://github.com/jasonwhite/darg
-> https://github.com/SirTony/commando
etc.

std.json:

-> https://github.com/s-ludwig/std_data_json

std.math:

-> https://code.dlang.org/packages/ctstdmath
-> https://github.com/libmir/mir-math

std.net.curl:

-> https://github.com/ikod/dlang-requests

std.net.isemail:

-> https://github.com/anton-dutov/mail
-> http://vibed.org/api/vibe.mail.smtp/Mail

std.signals:

-> https://code.dlang.org/packages/phobosx
-> https://code.dlang.org/packages/observe

std.socket:

-> http://vibed.org/api/vibe.http.websockets/
-> http://vibed.org/api/vibe.core.net/TCPConnection

std.stdio:

-> https://github.com/jasonwhite/io
-> https://github.com/schveiguy/iopipe
-> 
https://github.com/rejectedsoftware/vibe.d/blob/master/stream/vibe/stream/stdio.d


std.uri:

-> 
https://github.com/rikkimax/alphaPhobos/blob/master/source/std/experimental/uri.d

-> http://vibed.org/api/vibe.inet.url/
-> https://github.com/adamdruppe/arsd/blob/master/http2.d

std.variant

-> https://github.com/s-ludwig/taggedalgebraic

std.xml

-> https://github.com/lodo1995/experimental.xml
-> https://github.com/jacob-carlborg/orange


Also a lot of proposed Phobos modules already exist in the 
community:


std.color

https://github.com/TurkeyMan/color

std.decimal

https://github.com/andersonpd/eris/
https://github.com/jaypha/fixed

std.events (planned?)

-> https://github.com/etcimon/libasync


Of course many of these libraries are one man projects and aren't 
considered "high-quality", but the reason they exist in the first 
place shows that a lot of things just shouldn't be 
"standardized", but more provided with a "practical variant". We 
should focus solely on modules that can be standardized easily 
(std.traits, std.ranges, ...) or are required to create a unified 
experience (e.g. a common allocation/logging interface). Imho 
there's really no need to ship sth. like SQLite as part of the 
standard library.


Final remarks
-

- From a technical point of view there shouldn't be a problem to 
have a unified documentation experience etc.
- Thanks to Martin's awesome work, we already test a couple of 
"selected" DUB packages on every DMD/Druntime/Phobos commit


Re: A betterC modular standard library?

2016-12-18 Thread Ilya Yaroshenko via Digitalmars-d
On Sunday, 18 December 2016 at 20:16:11 UTC, Andrei Alexandrescu 
wrote:

On 12/18/2016 03:10 PM, Ilya Yaroshenko wrote:
On Sunday, 18 December 2016 at 19:28:24 UTC, Andrei 
Alexandrescu wrote:

On 12/18/2016 01:49 PM, Ilya Yaroshenko wrote:
On Sunday, 18 December 2016 at 17:41:31 UTC, Walter Bright 
wrote:

On 12/18/2016 1:26 AM, Ilya Yaroshenko wrote:
We already have better `cpuid` and better `random` 
packages.


Great! Please PR them for Phobos.


cpuid is used in Mir GLAS and it should be a betterC library.


It's this kind of imaginary dialog that I don't quite grok:

Ilya Yaroshenko: "I can't use druntime's cpuid so I defined 
my own."

[...]
IY: "..."

It's this kind of stuff I need to have a better understanding 
of. Some
technical arguments are meaningful, some others point to 
problems with
obvious solutions that are somehow shunned, and yet others 
are like a
hidden Markov model - I see the effects, but I don't see the 
causes.



Andrei



My last argument was:

Linking compatibility. Assume A has builded a betterC library 
AA using
GDC version XXX and B has builded a betterC library BB using 
LDC version

YYY.

Question: how mister C can use BB and AA together in his 
C-library if
both this libraries depends on different DRuntimes (compiler 
and version)?




My understanding is that if cpuid_v2.d does not pull any 
dependency, it doesn't matter whether it sits in druntime or 
elsewhere. Is that the case?



DRuntime is large and evaluates fast.


So is it moving too slow or too fast?


It is too slow as library to contribute to and too fast as 
standard library to depend on.



We have not backward binary
compatibility in DRuntime and we have not binary compatibility 
between

different compiler versions.


Even if we don't, we can guarantee that core.cpuid_v2 adds no 
link-time dependency. We may offer it as a pure template 
library, or as C APIs. Whatever floats your boat. Is this 
enough for ensuring compatibility?


I need to understand how it is can be done because I did not find 
a valid solution.


mir-cpuid has global symbols, they are accessed using extern(C) 
API. If multiple libraries trying to initialize it than it will 
be initialized only once.


extern C API does not solve a problem with DRuntime because 
current DRuntime is not a betterC library and it is huge. A huge 
betterC library is a problem too because it is less portable.


Anyway, if you find a proper solution please share it. How it can 
be implemented as template library? Does different libraries will 
have their own cpuid instantiations and __gshared symbols?


How modularity can break existing Phobos? Why not to allow new 
packages like mir-cpuid and std.allocator to be a betterC library 
with their own repository? This will not break existing Phobos 
and Druntime but this will allow me to write server and numeric 
software without copy-pasting Phobos codes. I prefer to fork 
something rather then copypast.


My plans to work on:
1. Low-level numeric libraries
2. Low-level fast async I/O adaptive generic event loop, 
asymmetric green threads, http2

3. Microelectromechanical systems

They will require new threads, new timers, new mutexes, new event 
loop (existing C libs are slow), to be used.


Existing one does not satisfy my requirements.

I do not earn any money for this work.

I am working on this projects because I believe in D and believe 
that my current and future clients will believe in D much more 
then now. I know what they need.


Ilya



Re: A betterC modular standard library?

2016-12-18 Thread Andrei Alexandrescu via Digitalmars-d

On 12/18/2016 03:10 PM, Ilya Yaroshenko wrote:

On Sunday, 18 December 2016 at 19:28:24 UTC, Andrei Alexandrescu wrote:

On 12/18/2016 01:49 PM, Ilya Yaroshenko wrote:

On Sunday, 18 December 2016 at 17:41:31 UTC, Walter Bright wrote:

On 12/18/2016 1:26 AM, Ilya Yaroshenko wrote:

We already have better `cpuid` and better `random` packages.


Great! Please PR them for Phobos.


cpuid is used in Mir GLAS and it should be a betterC library.


It's this kind of imaginary dialog that I don't quite grok:

Ilya Yaroshenko: "I can't use druntime's cpuid so I defined my own."
[...]
IY: "..."

It's this kind of stuff I need to have a better understanding of. Some
technical arguments are meaningful, some others point to problems with
obvious solutions that are somehow shunned, and yet others are like a
hidden Markov model - I see the effects, but I don't see the causes.


Andrei



My last argument was:

Linking compatibility. Assume A has builded a betterC library AA using
GDC version XXX and B has builded a betterC library BB using LDC version
YYY.

Question: how mister C can use BB and AA together in his C-library if
both this libraries depends on different DRuntimes (compiler and version)?



My understanding is that if cpuid_v2.d does not pull any dependency, it 
doesn't matter whether it sits in druntime or elsewhere. Is that the case?



DRuntime is large and evaluates fast.


So is it moving too slow or too fast?


We have not backward binary
compatibility in DRuntime and we have not binary compatibility between
different compiler versions.


Even if we don't, we can guarantee that core.cpuid_v2 adds no link-time 
dependency. We may offer it as a pure template library, or as C APIs. 
Whatever floats your boat. Is this enough for ensuring compatibility?




Andrei


Re: A betterC modular standard library?

2016-12-18 Thread Ilya Yaroshenko via Digitalmars-d
On Sunday, 18 December 2016 at 19:28:24 UTC, Andrei Alexandrescu 
wrote:

On 12/18/2016 01:49 PM, Ilya Yaroshenko wrote:
On Sunday, 18 December 2016 at 17:41:31 UTC, Walter Bright 
wrote:

On 12/18/2016 1:26 AM, Ilya Yaroshenko wrote:

We already have better `cpuid` and better `random` packages.


Great! Please PR them for Phobos.


cpuid is used in Mir GLAS and it should be a betterC library.


It's this kind of imaginary dialog that I don't quite grok:

Ilya Yaroshenko: "I can't use druntime's cpuid so I defined my 
own." [...]

IY: "..."

It's this kind of stuff I need to have a better understanding 
of. Some technical arguments are meaningful, some others point 
to problems with obvious solutions that are somehow shunned, 
and yet others are like a hidden Markov model - I see the 
effects, but I don't see the causes.



Andrei



My last argument was:

Linking compatibility. Assume A has builded a betterC library AA 
using GDC version XXX and B has builded a betterC library BB 
using LDC version YYY.


Question: how mister C can use BB and AA together in his 
C-library if both this libraries depends on different DRuntimes 
(compiler and version)?



DRuntime is large and evaluates fast. We have not backward binary 
compatibility in DRuntime and we have not binary compatibility 
between different compiler versions.


mir-glas is a betterC library for all languages.  It depends in 
mir-cpuid. I want mir-glas to replace OpenBLAS. To do it I need 
the same portability.


OK, assume we already have DRuntime for all targets including 
MIPS, POWER, armv7, aarch64, Loongson.


Does this solves problem? No.

The reason is very simple. All non-betterC D libraries must be 
compiled in lockstep: with the same compiler vendor and druntime 
version!


This will just kill GLAS. How this can be packed for Ubuntu for 
example? It will depends on LDC libphobos == XXX. Another numeric 
library, say FFT, will depend on libphobos == YYY by GDC. How 
they can be linked if they require different DRuntime from 
different compilers? Or different druntime versions for the same 
compiler?


Do you want to constraint D users to do not distribute binary 
libraries in popular repositories like Debian/Ubuntu?


DRuntime incompatibility is a huge constraints for business. Do 
you want to force all companies to use identical version of 
compiler and identical vendor?


Yes, C++ changes their API, but it few orders of magnitude slower 
then D. And they support multiple branches with bugs fixes. D 
supports only the recent one.


This is why GLAS/cpuid need to be a betterC library.

Ilya



Re: A betterC modular standard library?

2016-12-18 Thread Andrei Alexandrescu via Digitalmars-d

On 12/18/2016 01:49 PM, Ilya Yaroshenko wrote:

On Sunday, 18 December 2016 at 17:41:31 UTC, Walter Bright wrote:

On 12/18/2016 1:26 AM, Ilya Yaroshenko wrote:

We already have better `cpuid` and better `random` packages.


Great! Please PR them for Phobos.


cpuid is used in Mir GLAS and it should be a betterC library.


It's this kind of imaginary dialog that I don't quite grok:

Ilya Yaroshenko: "I can't use druntime's cpuid so I defined my own."

Andrei Alexandrescu: "OK, so what's different?"

IY: "Mine has a few engineering improvements."

AA: "Cool, why don't you merge them into core.cpuid?"

IY: "Well mine doesn't have a shared static constructor, so it doesn't 
need a runtime to automatically call that library initialization 
function. User need to explicitly call an init function before using it."


AA: "I understand. Great, so how about this - we add your code to 
core.cpuid_v2 to druntime. Then we make all in core.cpuid to forward to 
it so there's no duplication. It all works out!"


IY: "No, I don't want to do that. It's still in druntime and I don't 
want druntime. I want betterC."


AA: "But it will be compilable with betterC and we can add unittests to 
make that happen. YOU HAVE MY SUPPORT. Let's do it."


IY: "No, I want to change it often. The deployment schedule of druntime 
is too slow."


AA: "How often do you need to change it? Is it that unstable?"

IY: "Um, not too often."

AA: "Then what is the matter? Are you worried about the IP of the 
engineering improvements you are making? Are you licensing this 
differently?"


IY: "No, it's for the most part similar to core.cpuid and the license 
will be also Boost."


AA: "Then what is the matter? Do you want me to wait until you release a 
stable mir.cpuid and copy it over with credit, per the Boost license, to 
core.cpuid_v2?"


IY: "..."

It's this kind of stuff I need to have a better understanding of. Some 
technical arguments are meaningful, some others point to problems with 
obvious solutions that are somehow shunned, and yet others are like a 
hidden Markov model - I see the effects, but I don't see the causes.



Andrei



Re: A betterC modular standard library?

2016-12-18 Thread qznc via Digitalmars-d

On Sunday, 18 December 2016 at 14:37:55 UTC, Radu wrote:

On Sunday, 18 December 2016 at 09:26:09 UTC, Ilya Yaroshenko

Who is interested in betterC _modular_* standard library?
I am planing to make libmir org a community for it.
Thought and concerns?


I hope this doesn't turn into a new Tango vs. Phobos war.


My first thought as well. Probably an irrational fear, but the 
question will pop up often.


Re: A betterC modular standard library?

2016-12-18 Thread Ilya Yaroshenko via Digitalmars-d

On Sunday, 18 December 2016 at 17:41:31 UTC, Walter Bright wrote:

On 12/18/2016 1:26 AM, Ilya Yaroshenko wrote:

We already have better `cpuid` and better `random` packages.


Great! Please PR them for Phobos.


cpuid is used in Mir GLAS and it should be a betterC library. 
Without DRuntime dependency because DRuntime has not backward 
binary compatibility and has not compatibility between different 
compilers.


From my reply to Andrei:


Linking compatibility. Assume A has builded a betterC library AA 
using GDC version XXX and B has builded a betterC library BB 
using LDC version YYY.


Question: how mister C can use BB and AA together in his 
C-library if both this libraries depends on different DRuntimes 
(compiler and version)?



mir-glas and mir-cpuid are libraries with extern(C) interface for 
all languages. It is silly to build a BLAS implementation only 
for one language.



The betterC
std.range and std.algorithm analogs would be released with new 
ndslice
implementation. Mir's algorithm would be faster then Phobos 
and will generate
less template bloat. Then lightweight threads, multithread 
GLAS, matrix

inversion. Fastest I/O and http2 ...

(they all are betterC modular libraries)


Propose them for Phobos!


First I need to implement them and announce for the community. 
Then community will say "What he is doing??? C++ iterators, 
really?" Hehe


This is reason why ndslice is deprecated in Phobos. It requires 
another std.algorithm and std.range. I am not able to implement 
all this ideas in Phobos PRs: it is requires too much time. In 
addition, I can not use Phobos for other reasons.


Andrei and probably 90% of existing D users don't want Phobos 
to move this

direction.


I don't understand that point. What direction?


1. Modularity: D should provide a very slim library (like 
std.traits and may be default GC with old core, which can be 
turned off or replaced). Other parts or Phobos/Druntime should be 
split into parts and the parts should have their own 
repositories. They should be dub packages with D Foundation 
review / control.


2. Low-level API first! A high level OOP/GC-oriented API can be 
provided later or can be provided in DUB libraries.


3. betterC guaranties / No problems with backward binary 
compatibility for betterC libs. This means that code should be 
templated / inlined. Some parts, like Exceptions and GC may have 
this problems, but Exceptions and GC should be optional language 
features and they are not a part of betterC mode.


In other hand I need a commercial attractive D infrastructure 
for large and
heavy system projects. There is no commercial perspective for 
me to contribute

to Phobos because:

1. Phobos version depends on compiler version. Delay with LDC 
release is too

large. It should / can be one day.


The language evolves over time, and the standard library must, 
too. It happens with every language.


Agreed. Because that it is more flexible to have a very slim std 
library with good set of community supported libraries.


This is a reason why almost all portable multi-language numeric 
libraries are written in C. C runtime is slim and evaluates 
very-very slow.


2. We can not do hot fixes in Phobos without compiling and 
deploying patched

libphobos. Mir's DUB package are much more flexible.

3. Dependencies should be clear. Modularity is a proper way 
for large std
library. In phobos everything integrated with everything. 
DRuntime -> Phobos
abstraction is weird for betterC because system modules can 
depends universal
algorithms, but universal algorithm are more portable if they 
have not system

dependencies.


Not sure what you mean. Algorithms in Phobos are not dependent 
on system idiosyncracies.


They are, for example they depends on RangeErrors. See
https://github.com/dlang/phobos/pull/4937

4. Phobos does not provide and will not provide betterC 
guaranties. If something
works for betterC mode now it may not work in the future 
release.


I've done a fair amount of work to remove such things. Clearly 
more can and should be done.


Generally, we do have a goal of making Phobos entirely "pay as 
you go" instead of being so interconnected (which is what 
betterC is really all about). We can definitely use help in 
improving that and helping it along.


Great. Why not to go to the proposed direction:
1 Modularity,
2 Low-level API first,
3 betterC guaranties?

Thank you,
Ilya


Re: A betterC modular standard library?

2016-12-18 Thread Ilya Yaroshenko via Digitalmars-d
On Sunday, 18 December 2016 at 15:18:37 UTC, Andrei Alexandrescu 
wrote:

On 12/18/16 4:26 AM, Ilya Yaroshenko wrote:

Hi,

Who is interested in betterC _modular_* standard library?
I am planing to make libmir org a community for it.
Thought and concerns?

We already have better `cpuid` and better `random` packages. 
The betterC
std.range and std.algorithm analogs would be released with new 
ndslice
implementation. Mir's algorithm would be faster then Phobos 
and will
generate less template bloat. Then lightweight threads, 
multithread

GLAS, matrix inversion. Fastest I/O and http2 ...

(they all are betterC modular libraries)

Andrei and probably 90% of existing D users don't want Phobos 
to move

this direction.


Please do not misconstrue what I said.

You have my support to improve the state of affairs with the D 
core runtime and standard library, and have had it from day one.


With regard to new functionality that supplants existing pieces 
such as cpuid_v2 or random_v2, definitely. We can and should 
add that and many other great things to phobos. Go for it.


With regard to avoiding to link druntime, we should build the 
argument better. Is all of druntime bad, or just parts of it?


DRuntime is good for applications. But writing multi-language 
libraries with DRuntime is bad idea. The last paragraph in [1] 
describes why DRuntime can not be used for GLAS.


I never said we should "remove GC" or "remove runtime". We just 
need to provide a clear way for betterC subset and modularise 
library. DRuntime and Phobos are well designed hight level 
libraries.


E.g. if something does not link the GC but does use vtables, is 
it good/bad, and to what extent?


Very good point. First, I don't see a real reason why we need 
DRuntime for vtables (classes and interfaces). LDC allows to use 
vtables without DRuntime (only with extern(C++), hehe). D classes 
can be constructed on stack or with allocators. The only problem 
is vtable initlization, which requires `typeid` (TypeInfo 
object). I am not a OOP expert, but it looks like we can have 
extern(C++) classes without DRuntime if we solve problem with 
initilization.


These issues are not entirely clear to me and it would be great 
if you explained them better. I definitely need to be better 
educated on this. What I do know is making "don't use druntime" 
a goal in and of itself is not the most useful way to frame the 
problem.


0. Linking compatibility. Assume A has builded a betterC library 
AA using GDC version XXX and B has builded a betterC library BB 
using LDC version YYY.


Question: how mister C can use BB and AA together in his 
C-library if both this libraries depends on different DRuntimes 
(compiler and version)?


1. Different mangling names for DMD and LDC. mir-cpuid provides 
extern(C) interface for all functionality except few very low 
level struct declarations. The general problem is that we do not 
have binary compatibility where we may have it. And it is not 
clear where we already have it.


2. Any function may throw an Error.

3. Any throwable is not a part of betterC. Phobos core team 
rejected approach to replace RangeErrors with asserts or another 
function declaration. Possible alternative is __range_assert() or 
a library function.


4. Threads are GC oriented:
1. They are classes (they can be structs like allocators)
2. They are integrated with GC. This requires additional 
computation time if I am not wrong.
3. They are not replaceable. Phobos multithread modules can 
not use something else.


5. Mutexes are GC oriented:
   ditto 1. and 3.

6. Intrinsics are compiler magic without specification. It is not 
clear what it is intrinsic, are they inlined, where their bodies 
are defined, how/if they works in CTFE. LLVM provides a list of 
intrinsics [3], this is very useful to write a betterC libraries.


... and many other ... They all are not clear for me too.

-
A solution for mister C:
Mister A and mister B should use a runtime which has either a 
small stable extern(C) interface or all its functions are 
inlined. All other functions can be a dub packages.

-

Proof:
cpuid: extern(C) interface with aliases.
a fork of aligned  mallocator for GLAS: all functions are always 
inlined.


With regard to changes that break the entire fabric of Phobos 
(such as: we must eliminate random-access ranges), this is 
definitely more like simple survival rather than things I want.


I don't want eliminate R-A-R now. But we can change our Range 
concept and provide alternative module as dub package, which will 
be announced as a future replacement (say, after 5 years). But 
duplicated sorting implementation with cursors (lightweight 
iterators) in the same module is weird. It is better to continue 
this discussion after the cursor based implementation will be 
ready for your review.


With regard to properly attributing credit for good work, I 
believe this is important and I commit to do anything 

Re: A betterC modular standard library?

2016-12-18 Thread Walter Bright via Digitalmars-d

On 12/18/2016 1:26 AM, Ilya Yaroshenko wrote:

We already have better `cpuid` and better `random` packages.


Great! Please PR them for Phobos.



The betterC
std.range and std.algorithm analogs would be released with new ndslice
implementation. Mir's algorithm would be faster then Phobos and will generate
less template bloat. Then lightweight threads, multithread GLAS, matrix
inversion. Fastest I/O and http2 ...

(they all are betterC modular libraries)


Propose them for Phobos!



Andrei and probably 90% of existing D users don't want Phobos to move this
direction.


I don't understand that point. What direction?



In other hand I need a commercial attractive D infrastructure for large and
heavy system projects. There is no commercial perspective for me to contribute
to Phobos because:

1. Phobos version depends on compiler version. Delay with LDC release is too
large. It should / can be one day.


The language evolves over time, and the standard library must, too. It happens 
with every language.




2. We can not do hot fixes in Phobos without compiling and deploying patched
libphobos. Mir's DUB package are much more flexible.

3. Dependencies should be clear. Modularity is a proper way for large std
library. In phobos everything integrated with everything. DRuntime -> Phobos
abstraction is weird for betterC because system modules can depends universal
algorithms, but universal algorithm are more portable if they have not system
dependencies.


Not sure what you mean. Algorithms in Phobos are not dependent on system 
idiosyncracies.




4. Phobos does not provide and will not provide betterC guaranties. If something
works for betterC mode now it may not work in the future release.


I've done a fair amount of work to remove such things. Clearly more can and 
should be done.


Generally, we do have a goal of making Phobos entirely "pay as you go" instead 
of being so interconnected (which is what betterC is really all about). We can 
definitely use help in improving that and helping it along.


Re: A betterC modular standard library?

2016-12-18 Thread Ilya Yaroshenko via Digitalmars-d

On Sunday, 18 December 2016 at 15:20:48 UTC, Radu wrote:
2. In the same time many thing is not possible to create with 
Phobos/Druntime. The simplest example is numeric libraries. I 
can create a multithread GLAS with core.thread, but if I do so 
GLAS will never be accepted as default BLAS implementation for 
Julia. So, i need to create my own low level thread library. 
Intel MKL, OpenBLAS have their own thread management: this is 
common practice.


Cheers,
Ilya


I get point 2, but I should be able to use your new modules 
without giving up druntime/phobos, right?. They should not be 
mutual exclusive!


Yes, you are able to use any Mir libs with Phobos/DRuntime --Ilya


Re: A betterC modular standard library?

2016-12-18 Thread bachmeier via Digitalmars-d

On Sunday, 18 December 2016 at 15:20:48 UTC, Radu wrote:


I get point 2, but I should be able to use your new modules 
without giving up druntime/phobos, right?. They should not be 
mutual exclusive!


I think that being able to use them the same as any other C 
library should be sufficient.


Re: A betterC modular standard library?

2016-12-18 Thread Radu via Digitalmars-d
On Sunday, 18 December 2016 at 15:01:55 UTC, Ilya Yaroshenko 
wrote:

On Sunday, 18 December 2016 at 14:37:55 UTC, Radu wrote:

[...]


Maybe I will be able to propose them on DConf.


[...]


Yes/No in the same time.

1. It would be always possible to use Mir and Phobos together, 
100% guarantee. Furthermore, some Mir libraries are going to be 
infrastructure for _all_, not only for D. For example Mir GLAS 
will target the Julia language after D.


2. In the same time many thing is not possible to create with 
Phobos/Druntime. The simplest example is numeric libraries. I 
can create a multithread GLAS with core.thread, but if I do so 
GLAS will never be accepted as default BLAS implementation for 
Julia. So, i need to create my own low level thread library. 
Intel MKL, OpenBLAS have their own thread management: this is 
common practice.


Cheers,
Ilya


I get point 2, but I should be able to use your new modules 
without giving up druntime/phobos, right?. They should not be 
mutual exclusive!


Re: A betterC modular standard library?

2016-12-18 Thread Andrei Alexandrescu via Digitalmars-d

On 12/18/16 4:26 AM, Ilya Yaroshenko wrote:

Hi,

Who is interested in betterC _modular_* standard library?
I am planing to make libmir org a community for it.
Thought and concerns?

We already have better `cpuid` and better `random` packages. The betterC
std.range and std.algorithm analogs would be released with new ndslice
implementation. Mir's algorithm would be faster then Phobos and will
generate less template bloat. Then lightweight threads, multithread
GLAS, matrix inversion. Fastest I/O and http2 ...

(they all are betterC modular libraries)

Andrei and probably 90% of existing D users don't want Phobos to move
this direction.


Please do not misconstrue what I said.

You have my support to improve the state of affairs with the D core 
runtime and standard library, and have had it from day one.


With regard to new functionality that supplants existing pieces such as 
cpuid_v2 or random_v2, definitely. We can and should add that and many 
other great things to phobos. Go for it.


With regard to avoiding to link druntime, we should build the argument 
better. Is all of druntime bad, or just parts of it? E.g. if something 
does not link the GC but does use vtables, is it good/bad, and to what 
extent? These issues are not entirely clear to me and it would be great 
if you explained them better. I definitely need to be better educated on 
this. What I do know is making "don't use druntime" a goal in and of 
itself is not the most useful way to frame the problem.


With regard to changes that break the entire fabric of Phobos (such as: 
we must eliminate random-access ranges), this is definitely more like 
simple survival rather than things I want.


With regard to properly attributing credit for good work, I believe this 
is important and I commit to do anything reasonable for making it happen.


I kindly advise you to better understand what you are trying to do, and 
how to do it to maximize impact. It seems you are not entirely clear on 
what you want to do and what the best route is, as shown by you retiring 
your std.experimental.ndslice work after you've had full freedom on what 
should go in it.



In other hand I need a commercial attractive D infrastructure for large
and heavy system projects. There is no commercial perspective for me to
contribute to Phobos because:

1. Phobos version depends on compiler version. Delay with LDC release is
too large. It should / can be one day.


Then a good way to help this is improve the LDC lag.


2. We can not do hot fixes in Phobos without compiling and deploying
patched libphobos. Mir's DUB package are much more flexible.


The reverse of that is less stability. The good part about merging your 
work in Phobos is you maximize impact and visibility.


These seem to me artificial arguments. Yes, being preoccupied over quick 
deployment may make sense in the initial stages when there's no 
stability and the design may take any direction. That should get less 
necessary once some abstractions are in place.


Organizations that are interested in patching minute fixes may build 
their own standard library (and sometimes compiler). This is routinely 
the case with C++ at Facebook, Google, and in all likelihood other 
large-scale tech shops. It would be tenuous to argue that this should be 
the default way to go about things for everyone. I don't understand why 
you consider it an essential matter.



3. Dependencies should be clear. Modularity is a proper way for large
std library. In phobos everything integrated with everything. DRuntime
-> Phobos abstraction is weird for betterC because system modules can
depends universal algorithms, but universal algorithm are more portable
if they have not system dependencies.


I read this three times and cannot understand it. At any rate: DIP 1005, 
which is partially motivated by our discussions, should allow us to make 
dependencies as simple and clean as function-level.



4. Phobos does not provide and will not provide betterC guaranties. If
something works for betterC mode now it may not work in the future release.


Yes, we can definitely improve this.

We (the Foundation) are all for making things better for achieving the 
things you want to achieve. At this point D needs unity and rallying 
under a shared vision, not balkanization in several special interest 
groups. There are so many great things you can do starting literally 
today to make things better for everybody using D, and why in the world 
would anyone want to stop you. I strongly believe your participation to 
the Standard D language and library will be vastly better for you 
personally and professionally.



Andrei



Re: A betterC modular standard library?

2016-12-18 Thread Ilya Yaroshenko via Digitalmars-d

On Sunday, 18 December 2016 at 14:37:55 UTC, Radu wrote:
On Sunday, 18 December 2016 at 09:26:09 UTC, Ilya Yaroshenko 
wrote:

Hi,

Who is interested in betterC _modular_* standard library?
I am planing to make libmir org a community for it.
Thought and concerns?

We already have better `cpuid` and better `random` packages. 
The betterC std.range and std.algorithm analogs would be 
released with new ndslice implementation. Mir's algorithm 
would be faster then Phobos and will generate less template 
bloat. Then lightweight threads, multithread GLAS, matrix 
inversion. Fastest I/O and http2 ...


(they all are betterC modular libraries)

Andrei and probably 90% of existing D users don't want Phobos 
to move this direction.


In other hand I need a commercial attractive D infrastructure 
for large and heavy system projects. There is no commercial 
perspective for me to contribute to Phobos because:


1. Phobos version depends on compiler version. Delay with LDC 
release is too large. It should / can be one day.


2. We can not do hot fixes in Phobos without compiling and 
deploying patched libphobos. Mir's DUB package are much more 
flexible.


3. Dependencies should be clear. Modularity is a proper way 
for large std library. In phobos everything integrated with 
everything. DRuntime -> Phobos abstraction is weird for 
betterC because system modules can depends universal 
algorithms, but universal algorithm are more portable if they 
have not system dependencies.


4. Phobos does not provide and will not provide betterC 
guaranties. If something works for betterC mode now it may not 
work in the future release.


[1] https://github.com/libmir/
* - separate github/DUB libraries with clear dependencies.

Best regards,
Ilya


I hope this doesn't turn into a new Tango vs. Phobos war.

If you have ideas on how to improve things without messing the 
entire community with ego wars and such, then it sounds good. 
Otherwise thumbs down!


Maybe I will be able to propose them on DConf.

The first sign of faith will be that your modules will always 
work with druntime and phobos - i.e. you wont require a special 
runtime for your stuff to work.


Yes/No in the same time.

1. It would be always possible to use Mir and Phobos together, 
100% guarantee. Furthermore, some Mir libraries are going to be 
infrastructure for _all_, not only for D. For example Mir GLAS 
will target the Julia language after D.


2. In the same time many thing is not possible to create with 
Phobos/Druntime. The simplest example is numeric libraries. I can 
create a multithread GLAS with core.thread, but if I do so GLAS 
will never be accepted as default BLAS implementation for Julia. 
So, i need to create my own low level thread library. Intel MKL, 
OpenBLAS have their own thread management: this is common 
practice.


Cheers,
Ilya


Re: A betterC modular standard library?

2016-12-18 Thread Radu via Digitalmars-d
On Sunday, 18 December 2016 at 09:26:09 UTC, Ilya Yaroshenko 
wrote:

Hi,

Who is interested in betterC _modular_* standard library?
I am planing to make libmir org a community for it.
Thought and concerns?

We already have better `cpuid` and better `random` packages. 
The betterC std.range and std.algorithm analogs would be 
released with new ndslice implementation. Mir's algorithm would 
be faster then Phobos and will generate less template bloat. 
Then lightweight threads, multithread GLAS, matrix inversion. 
Fastest I/O and http2 ...


(they all are betterC modular libraries)

Andrei and probably 90% of existing D users don't want Phobos 
to move this direction.


In other hand I need a commercial attractive D infrastructure 
for large and heavy system projects. There is no commercial 
perspective for me to contribute to Phobos because:


1. Phobos version depends on compiler version. Delay with LDC 
release is too large. It should / can be one day.


2. We can not do hot fixes in Phobos without compiling and 
deploying patched libphobos. Mir's DUB package are much more 
flexible.


3. Dependencies should be clear. Modularity is a proper way for 
large std library. In phobos everything integrated with 
everything. DRuntime -> Phobos abstraction is weird for betterC 
because system modules can depends universal algorithms, but 
universal algorithm are more portable if they have not system 
dependencies.


4. Phobos does not provide and will not provide betterC 
guaranties. If something works for betterC mode now it may not 
work in the future release.


[1] https://github.com/libmir/
* - separate github/DUB libraries with clear dependencies.

Best regards,
Ilya


I hope this doesn't turn into a new Tango vs. Phobos war.

If you have ideas on how to improve things without messing the 
entire community with ego wars and such, then it sounds good. 
Otherwise thumbs down!


The first sign of faith will be that your modules will always 
work with druntime and phobos - i.e. you wont require a special 
runtime for your stuff to work.


As for Andrei and Walter - I think you need to see why people 
(talented people!) want to side walk the phobos & druntime normal 
contribution routes. Something clearly is wrong there!


Re: A betterC modular standard library?

2016-12-18 Thread Ilya Yaroshenko via Digitalmars-d

On Sunday, 18 December 2016 at 11:06:58 UTC, Temtaime wrote:

Let's write a D, phobos.
Then we drop all the features and use a BetterC mode.
Then we write all modules against betterC mode.


Sounds like a plan! --Ilya


Re: A betterC modular standard library?

2016-12-18 Thread Temtaime via Digitalmars-d
On Sunday, 18 December 2016 at 09:26:09 UTC, Ilya Yaroshenko 
wrote:

Hi,

Who is interested in betterC _modular_* standard library?
I am planing to make libmir org a community for it.
Thought and concerns?

We already have better `cpuid` and better `random` packages. 
The betterC std.range and std.algorithm analogs would be 
released with new ndslice implementation. Mir's algorithm would 
be faster then Phobos and will generate less template bloat. 
Then lightweight threads, multithread GLAS, matrix inversion. 
Fastest I/O and http2 ...


(they all are betterC modular libraries)

Andrei and probably 90% of existing D users don't want Phobos 
to move this direction.


In other hand I need a commercial attractive D infrastructure 
for large and heavy system projects. There is no commercial 
perspective for me to contribute to Phobos because:


1. Phobos version depends on compiler version. Delay with LDC 
release is too large. It should / can be one day.


2. We can not do hot fixes in Phobos without compiling and 
deploying patched libphobos. Mir's DUB package are much more 
flexible.


3. Dependencies should be clear. Modularity is a proper way for 
large std library. In phobos everything integrated with 
everything. DRuntime -> Phobos abstraction is weird for betterC 
because system modules can depends universal algorithms, but 
universal algorithm are more portable if they have not system 
dependencies.


4. Phobos does not provide and will not provide betterC 
guaranties. If something works for betterC mode now it may not 
work in the future release.


[1] https://github.com/libmir/
* - separate github/DUB libraries with clear dependencies.

Best regards,
Ilya


Let's write a D, phobos.
Then we drop all the features and use a BetterC mode.
Then we write all modules against betterC mode.