Re: hope we have the distributed computing perl6

2021-11-29 Thread yonghua via perl6-users
Y‌es it is exactly. thanks for updating this.   De : "William Michels via perl6-users" A : "rakoons" ,"Piper H" ,yong...@laposte.net Envoyé: mardi 30 Novembre 2021 09:51 Objet : Re: hope we have the distributed computing perl6   Hello Yonghua! I tried the link yo

Re: hope we have the distributed computing perl6

2021-11-29 Thread William Michels via perl6-users
t; > A : "William Michels" ,"rakoons" > Envoyé: mardi 30 Novembre 2021 09:13 > Objet : Re: hope we have the distributed computing perl6 > > > > On Tue, Nov 30, 2021 at 1:38 AM William Michels > wrote: > >> Hi Piper, >> >> RE: >>

Re: hope we have the distributed computing perl6

2021-11-29 Thread yonghua via perl6-users
I have seen this interesting article for perl vs python's distributed computing. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.462.1737&rep=rep1&type=pdf‌ Regards.   De : "Piper H" A : "William Michels" ,"rakoons" Envoyé: mardi 30 Novem

Re: hope we have the distributed computing perl6

2021-11-29 Thread Piper H
On Tue, Nov 30, 2021 at 1:38 AM William Michels wrote: > Hi Piper, > > RE: > > > I have copied some active members of the Perl community on this email, > in the hopes that they can help transfer the "perl-spark" Github > project. > > Best Regards, Bill. > > Thank you Bill. If there is a perl ba

Re: hope we have the distributed computing perl6

2021-11-29 Thread William Michels via perl6-users
Hi Piper, RE: https://github.com/perl-spark Thank you for the reply. There seems to be two issues here: 1) 'What is going on with Perl-Spark?' and 2). 'Can we make an effort to produce Raku-Spark?'. Below I only address the former question. The "perl-spark" Github project appears to contain 13 r

Re: hope we have the distributed computing perl6

2021-11-29 Thread Piper H
Hello Simon, yes I know RMQ quite well. But I don't think perl with RMQ will provide the functions of distribution. Maybe Apache Pulsar can do that. Pulsar has a good Websocket API, based on that a perl client will be implemented easily. But this project is still on startup. I have tested it, it's

Re: hope we have the distributed computing perl6

2021-11-28 Thread Simon Proctor
I've been watching this for a bit and was thinking that a combination of a message bus (Rabbit MQ?) and Cro should provide most of what you'd need for a backbone. The fact that Raku has Supplies and Channels built in means it feels like a problem that's easy enough to fix. This is probably me com

Re: hope we have the distributed computing perl6

2021-11-28 Thread Piper H
William, I didn't use SparkR. I use R primarily for plotting. Spark's basic API is quite simple, it does the distributed computing of map, filter, group, reduce etc, which are all covered by perl's map, sort, grep functions IMO. for instance, this common statistics on Spark: >>> fruit.take(5) [(

Re: hope we have the distributed computing perl6

2021-11-28 Thread William Michels via perl6-users
Hi Piper! Have you used SparkR (R on Spark)? https://spark.apache.org/docs/latest/sparkr.html I'm encouraged by the data-type mapping between R and Spark. It suggests to me that with a reasonable Spark API, mapping data types between Raku and Spark should be straightforward: https://spark.apach

hope we have the distributed computing perl6

2021-11-27 Thread Piper H
I use perl5 everyday for data statistics. The scripts are running on a single server for the computing tasks. I also use R, which has the similar usage. When we face very large data, we change to Apache Spark for distributed computing. Spark's interface languages (python, scala, even ruby) are not