Re: Netflix opensources its first D library: Vectorflow
On Tuesday, 8 August 2017 at 18:51:26 UTC, Nordlöw wrote: To clarify here's an incomplete snippet that should clarify: auto add(A, B)(A a, B b) should be auto add(A, B)(auto ref A a, auto ref B b)
Re: Netflix opensources its first D library: Vectorflow
On 10/18/2017 1:49 PM, Stephan Dilly wrote: On 2017-08-02 21:31:19 +, Walter Bright said: https://www.reddit.com/r/programming/comments/6r6dwp/netflix_opensources_its_first_d_library_vectorflow/ D got another mention in Netflix's popular tech blog: https://medium.com/netflix-techblog/machine-learning-platform-meetup-ddec090f3c17 (+meetup slides) --Stephan The quote: "VectorFlow was designed for these contexts and this philosophy informed several design decisions. For example the choice of the language, D, was one such decision. The modern system language was chosen to address the goal of a single language in adhoc as well as production contexts — providing the power of C++ and the simplicity/clarity of Python."
Re: Netflix opensources its first D library: Vectorflow
On 2017-08-02 21:31:19 +, Walter Bright said: https://www.reddit.com/r/programming/comments/6r6dwp/netflix_opensources_its_first_d_library_vectorflow/ D got another mention in Netflix's popular tech blog: https://medium.com/netflix-techblog/machine-learning-platform-meetup-ddec090f3c17 (+meetup slides) --Stephan
Re: Netflix opensources its first D library: Vectorflow
On Tuesday, 8 August 2017 at 18:40:08 UTC, Nordlöw wrote:
On Thursday, 3 August 2017 at 04:40:05 UTC, Matt wrote:
Also note, one of the main advantages of Eigen is the whole
lazy evaluation of expressions for compound operations.
I haven't dug in the source, but it's my understanding it's
done through a lot of compile time C++ template hacking
Note that D provides
__traits(isRef, Symbol)
To clarify here's an incomplete snippet that should clarify:
auto add(A, B)(A a, B b)
if (isSomeArithmeticType!A &&
isSomeArithmeticType!B)
{
static if (__traits(isRef, a) &&
__traits(isRef, b)) // both `a` and `b` are l-values
{
return a + b; // fully eager evaluation
}
else static if (__traits(isRef, a)) // `b` is an r-value
{
// `b` can incremented by `a` and returned by move (reused)
}
else static if (__traits(isRef, b)) // `a` is an r-value
{
// `a` can incremented by `b` and returned by move (reused)
}
else // both `a` and `b` are r-values
{
return Add(a,b); // delay evaluation
}
}
Re: Netflix opensources its first D library: Vectorflow
On Thursday, 3 August 2017 at 04:40:05 UTC, Matt wrote: Also note, one of the main advantages of Eigen is the whole lazy evaluation of expressions for compound operations. I haven't dug in the source, but it's my understanding it's done through a lot of compile time C++ template hacking Note that D provides __traits(isRef, Symbol) which can be used to convenient implement lazy-evaluated expressions with free functions. Note that it cannot currently be used to check whether `this` was passed as an l-value or r-value which prevents the C++-expression-template-pattern from being used in operator overloading. For details see: https://issues.dlang.org/show_bug.cgi?id=17734
Re: Netflix opensources its first D library: Vectorflow
On Wednesday, 2 August 2017 at 22:56:32 UTC, Joakim wrote: On Wednesday, 2 August 2017 at 21:31:19 UTC, Walter Bright wrote: https://www.reddit.com/r/programming/comments/6r6dwp/netflix_opensources_its_first_d_library_vectorflow/ No. 2 liked proggit link of the day, should be no. 1 soon: https://www.reddit.com/r/programming/top/?time=day Not doing well on HN though: https://hn.algolia.com/?query=vectorflow Top 3 for the week: https://www.reddit.com/r/programming/top/?sort=top=week People seem really enthused by this library.
Re: Netflix opensources its first D library: Vectorflow
On Thursday, 3 August 2017 at 14:00:31 UTC, Matt wrote: Meanwhile, the blog post Laeeth gave you shows Mir doing better on matrix multiplication benchmarks than Eigen, significantly better when dealing with complex numbers. I mean by now we should all be jaded enough not to simply take toy benchmarks as gospel for which is actually fastest in a non-trivial application. That's why I didn't make such a general claim and noted that the linked benchmarks only dealt with matrix multiplication. :P I don't doubt mir is really fast, though. Yes, the benchmarks are indicative, but it's up to you come up with a benchmark that characterizes your workload better.
Re: Netflix opensources its first D library: Vectorflow
On Wednesday, 2 August 2017 at 22:56:32 UTC, Joakim wrote: Not doing well on HN though: https://hn.algolia.com/?query=vectorflow HN is very sensitive to time of day when submitting. Did a new try: https://news.ycombinator.com/item?id=14920608
Re: Netflix opensources its first D library: Vectorflow
Meanwhile, the blog post Laeeth gave you shows Mir doing better on matrix multiplication benchmarks than Eigen, significantly better when dealing with complex numbers. I mean by now we should all be jaded enough not to simply take toy benchmarks as gospel for which is actually fastest in a non-trivial application. I don't doubt mir is really fast, though.
Re: Netflix opensources its first D library: Vectorflow
On Thursday, 3 August 2017 at 04:40:05 UTC, Matt wrote: Also note, one of the main advantages of Eigen is the whole lazy evaluation of expressions for compound operations. Yes, Mir does that too: http://blog.mir.dlang.io/ndslice/algorithm/optimization/2016/12/12/writing-efficient-numerical-code.html I haven't dug in the source, but it's my understanding it's done through a lot of compile time C++ template hacking Meanwhile, the blog post Laeeth gave you shows Mir doing better on matrix multiplication benchmarks than Eigen, significantly better when dealing with complex numbers.
Re: Netflix opensources its first D library: Vectorflow
On Thursday, 3 August 2017 at 04:40:05 UTC, Matt wrote: Also note, one of the main advantages of Eigen is the whole lazy evaluation of expressions for compound operations. I haven't dug in the source, but it's my understanding it's done through a lot of compile time C++ template hacking Yep, there's a lot of operator overloading and other C++ black magic. DCompute should be able to do some of the out of the box with the lambda kernels to get some data locality. I'm also in the process of integrating Polly (LLVM's polyhedral optimiser) into LDC to get great loop optimisations.
Re: Netflix opensources its first D library: Vectorflow
On Thursday, 3 August 2017 at 04:37:30 UTC, Matt wrote: That seems like a good start. I'll probably start contributing within a few weeks. Great, http://gitter.im/libmir is probably the best way to get in contact. In the long run the goal should probably be to have a "go-to" set of LA libraries, with appropriate data structures that have friendly bindings to GPU LA computations with DCompute. That is the idea, https://github.com/libmir/mir-glas already provides a BLAS and I'll eventually get around to getting DCompute integration for it and ndslice.
Re: Netflix opensources its first D library: Vectorflow
Also note, one of the main advantages of Eigen is the whole lazy evaluation of expressions for compound operations. I haven't dug in the source, but it's my understanding it's done through a lot of compile time C++ template hacking
Re: Netflix opensources its first D library: Vectorflow
That seems like a good start. I'll probably start contributing within a few weeks. In the long run the goal should probably be to have a "go-to" set of LA libraries, with appropriate data structures that have friendly bindings to GPU LA computations with DCompute.
Re: Netflix opensources its first D library: Vectorflow
On Thursday, 3 August 2017 at 03:46:11 UTC, Matt wrote: On Wednesday, 2 August 2017 at 21:31:19 UTC, Walter Bright wrote: https://www.reddit.com/r/programming/comments/6r6dwp/netflix_opensources_its_first_d_library_vectorflow/ Speakng of D in data science (where I think it can get traction), is there a standardized linear algebra library in D? Perhaps some hooks to Eigen? I saw a few upstarts LA libraries, but none that I would consider "the one to use in D", like numpy in python or Eigen in C++ We're using D in finance. D libraries are far from the maturity of Python, but they are developing quickly. https://github.com/kaleidicassociates/lubeck Library we have open-sourced https://github.com/libmir/mir-algorithm Mir library we sponsor Other mir libraries: https://github.com/libmir/mir-lapack https://github.com/libmir/mir-blas https://github.com/libmir/lapack https://github.com/DlangScience/cblas Performance matters: http://blog.mir.dlang.io/glas/benchmark/openblas/2016/09/23/glas-gemm-benchmark.html
Re: Netflix opensources its first D library: Vectorflow
On Wednesday, 2 August 2017 at 21:31:19 UTC, Walter Bright wrote: https://www.reddit.com/r/programming/comments/6r6dwp/netflix_opensources_its_first_d_library_vectorflow/ Speakng of D in data science (where I think it can get traction), is there a standardized linear algebra library in D? Perhaps some hooks to Eigen? I saw a few upstarts LA libraries, but none that I would consider "the one to use in D", like numpy in python or Eigen in C++
Re: Netflix opensources its first D library: Vectorflow
On Wednesday, 2 August 2017 at 21:31:19 UTC, Walter Bright wrote: https://www.reddit.com/r/programming/comments/6r6dwp/netflix_opensources_its_first_d_library_vectorflow/ No. 2 liked proggit link of the day, should be no. 1 soon: https://www.reddit.com/r/programming/top/?time=day Not doing well on HN though: https://hn.algolia.com/?query=vectorflow
Netflix opensources its first D library: Vectorflow
https://www.reddit.com/r/programming/comments/6r6dwp/netflix_opensources_its_first_d_library_vectorflow/
