On Monday, 15 February 2016 at 11:09:10 UTC, data pulverizer
wrote:
Are there are any plans to create a scala spark-like RDD class
for D
(https://www.cs.berkeley.edu/~matei/papers/2012/nsdi_spark.pdf)? This is a powerful model that has taken the data science world by storm; it would be useful to have something like this in the D world. Most of the algorithms in statistics/data science are iterative in nature which fits well with this kind of data model.
I read through the Kind Of Container thread which has some
relationship with this issue
(https://forum.dlang.org/thread/n07rh8$dmb$1...@digitalmars.com).
It looks like Immutability would be the way to go for an RDD
data structure. But I am not wedded to any model as long as we
can have something that performs the same functionality as the
RDD.
As an alternative are there plans for parallel/cluster
computing frameworks for D?
Apologies if I am kicking a hornet's nest. It is not my
intention.
Thanks
Perhaps the question is too prescriptive. Another way is: Does D
have a big data strategy? But I tried to anchor it to some
currently functioning framework which is why I suggested RDD.