Re: perl streaming framework

2020-07-14 Thread JJ Merelo
Hi, El mar., 14 jul. 2020 a las 4:35, Warren Pang () escribió: > Hi > > Does perl have a stream computing framework? > I don't really know about Perl. It used to have Perl Data Language, but I don't know its current state. However, this mailing list is about Raku, or Perl 6 as it was called

Re: perl streaming framework

2020-07-14 Thread Fernando Santagata
…or maybe something like Pandas https://pandas.pydata.org/ ? BTW, it would be nice to build something like that for Raku. (Shameless plug: I'm working on a Raku interface to the GNU Scientific Library (see: https://modules.raku.org/search/?q=Math%3A%3ALibgsl). The next modules in my queue are

Re: perl streaming framework

2020-07-14 Thread Richard Hainsworth
There was a talk by Steven Lembark about using the lazy gathers and concurrency constructs in Raku at the last Perl conference that might be of interest. Steven (if I remember correctly) was using these tools to analyse vast quantities of data. The language constructs exist in Raku, and the

Re: perl streaming framework

2020-07-14 Thread Shlomi Fish
Hi Warren! Please reply to list. On Tue, 14 Jul 2020 10:34:57 +0800 Warren Pang wrote: > Hi > > Does perl have a stream computing framework? > I know Java/python have many, such as spark, flink, beam etc. > But I am looking for a perl alternative, since most of our team members > have been

NativeLibs not loading on Module testing

2020-07-14 Thread Richard Hainsworth
I'm adding a Module to the ecosystem, and the Build system failed, see https://travis-ci.com/github/Raku/ecosystem/builds/175326153 It doesn't seem that it is my Module that's failing, but NativeLibs. see line 165. As for my Module (raku-pod-render) it passes all tests, including Test-Meta

Re: perl streaming framework

2020-07-14 Thread Warren Pang
Hello We are already using perl PDL and something similar. The streaming framework is mainly used for real time data analysis. Tyler from Apache Beam project has wrote a great book about streaming system: http://streamingsystems.net/ So I have interest to know if there is the perl implementation

Re: perl streaming framework

2020-07-14 Thread Brad Gilbert
If you really want a streaming framework for Perl, the mailing list for Raku users might not be the best place to ask. (Raku used to be known as Perl6, and we haven't done anything to change the name of this mailing list.) Raku has a very similar syntax to Perl. (It used to be called Perl6 after

impact of compiling pod doc

2020-07-14 Thread Marcel Timmerman
Hi, I was wondering if pod documentation has a large impact on compiling modules. I assume on small files it would be ignorable but I have some modules which have grown big, one has 7153 lines of which the biggest part is pod doc. Would it be useful to split such a file in two. one for code

Re: perl streaming framework

2020-07-14 Thread Shlomi Fish
Hi Warren, On Tue, 14 Jul 2020 16:54:49 +0800 Warren Pang wrote: > Hello > > We are already using perl PDL and something similar. > The streaming framework is mainly used for real time data analysis. > Tyler from Apache Beam project has wrote a great book about streaming > system: >

Re: perl streaming framework

2020-07-14 Thread Aureliano Guedes
I already mentioned it here. Since Raku supports truly well functional programming. To data analysis might be better something like https://www.tidyverse.org/ implemented in R to approximate a natural language. On Tue, Jul 14, 2020 at 11:03 AM Brad Gilbert wrote: > If you really want a

Re: perl streaming framework

2020-07-14 Thread Ralph Mellor
I second Brad's comment -- perlmonks.org is a great place to ask. Use the following link for a quick start. The user interface is something out of the dark ages but you'll generally get outstandingly good replies, especially if you explicitly add something like "Thank you for considering my

Re: NativeLibs not loading on Module testing

2020-07-14 Thread JJ Merelo
I've submitted now a PR to NativeLibs that fixes that issue. El mar., 14 jul. 2020 a las 11:03, Richard Hainsworth (< rnhainswo...@gmail.com>) escribió: > I'm adding a Module to the ecosystem, and the Build system failed, see > > https://travis-ci.com/github/Raku/ecosystem/builds/175326153 > >

Re: impact of compiling pod doc

2020-07-14 Thread JJ Merelo
It's going to be pretty much the same. If it's installed, it's going to be precompiled anyway. The overhead added by reading the file might even make it slower. Of course, you can always give it a try and measure. Measuring might always surprise you :-) El mar., 14 jul. 2020 a las 14:20, Marcel

Re: impact of compiling pod doc

2020-07-14 Thread Marcel Timmerman
Thank you for answer JJ. It is quite a work to split such a large file up so I thought I'd better ask before I dive in head first. Later I might think it over. I have this documentation on the github pages too so I could separate the pod doc and keep them somewhere else where they won't get

Re: perl streaming framework

2020-07-14 Thread Warren Pang
Thank you all very much. I have tried to search "perl discussion" and it brings me to perl6's list page. Yes for data analysis we primarily use classic perl5, which is smart enough especially the regex matching. There are "spark streaming", "flink streaming", "storm streaming", and a lot of

Re: perl streaming framework

2020-07-14 Thread William Michels via perl6-users
Steven Lembark's talk (using Raku to analyze 123GB of BLAST-formatted AA sequences). Definitely worth watching: https://www.youtube.com/watch?v=rgCk5w2o-GY On Tue, Jul 14, 2020 at 2:16 AM Richard Hainsworth wrote: > > There was a talk by Steven Lembark about using the lazy gathers and >

Re: perl streaming framework

2020-07-14 Thread Aureliano Guedes
I'm really sad about the conceptions of programming with non-functional languages. Truly better an R as a reference than a OO in data analysis. On Wed, Jul 15, 2020 at 12:13 AM William Michels wrote: > Hi Warren, > > The topic you raise is an interesting one. When I've previously > searched for

Re: perl streaming framework

2020-07-14 Thread William Michels via perl6-users
Hi Warren, The topic you raise is an interesting one. When I've previously searched for 'streaming' and 'Python' I often see articles on Iterables such as the following: [1] "Data streaming in Python: generators, iterators, iterables"

Re: perl streaming framework

2020-07-14 Thread Warren Pang
Thanks for the info @Wiliam. We are probably seeking for a streaming framework which is Engineering-oriented. The documentation you provided are mostly Algorithm-oriented. For instance, as an algorithm engineer, I can implement the algorithm Logistic regression by hand. But I won't do this,