Thanks for sharing the slides Rémi! That looks really cool. One question I have after this, do you plan to use S3 Select ( https://aws.amazon.com/blogs/aws/s3-glacier-select/)?Seems it would fit your architecture nicely and I think shouldn't be too hard to create the query from the filters/projection in the datasource scan method to spend less time in Lambda.
On Wed, Feb 10, 2021, 18:44 Rémi Dettai <rdet...@gmail.com> wrote: > Thanks for the notes Andy. Here is the slide deck I presented, for further > reference: > > https://docs.google.com/presentation/d/1uZ5PbazC1zCX24k0Hh-UItddIh9BRvD5GL7NUDgc9eQ/edit?usp=sharing > > If anyone wants to see how it works in practice and does not have an AWS > account to try it out, feel free to reach out to me and I can walk you > through it! > > Le mer. 10 févr. 2021 à 18:37, Andy Grove <andygrov...@gmail.com> a écrit > : > > > Attendees > > > > > > - > > > > Andy Grove > > - > > > > Benjamin Blodgett > > - > > > > Marc Prud’Hommeaux > > - > > > > Mike Seddon > > - > > > > Jorge Leitao > > - > > > > Andrew Lamb > > - > > > > Fernando Herrera > > - > > > > Neville Dipale > > - > > > > Remi Dettai > > > > > > (Please let me know if I have misspelled anyone’s names) > > > > Topics Discussed > > > > > > - > > > > Discussion of Jorge’s proposal to redesign Arrow crate to resolve > safety > > violations (following on from mailing list discussion) > > - > > > > Mike has a PR up to implement a large number of Postgres string > > functions that needs reviewing > > - > > > > Remi gave a short presentation about his Buzz project which provides > > serverless compute using Arrow and DataFusion > > > > > > Planned for next time: > > > > > > - > > > > Marc Prud’Hommeaux to give a presentation/demo on his use of Arrow > > - > > > > Andy Grove to give a presentation/demo on Ballista, which provides > > distributed query execution using DataFusion > > > > > > On Wed, Feb 10, 2021 at 8:56 AM Andy Grove <andygrov...@gmail.com> > wrote: > > > > > A quick reminder that the bi-weekly Arrow Rust sync call starts about > an > > > hour from now. Everyone is welcome. > > > > > > https://meet.google.com/ctp-yujs-aee > > > > > > Thanks, > > > > > > Andy. > > > > > >
