I would like to call out that Beam itself can be directly used for ETL, no extra framework required (not to say that both of these frameworks don't provide additional value, e.g. GUI-style construction of pipelines).
On Fri, Oct 11, 2019 at 9:29 AM Ryan Skraba <[email protected]> wrote: > > Hello! Talend has a big data ETL product in the cloud called Pipeline > Designer, entirely powered by Beam. There was a talk at Beam Summit > 2018 (https://www.youtube.com/watch?v=1AlEGUtiQek), but unfortunately > the live demo wasn't captured in the video. You can find other videos > of Pipeline Designer online to see if it might fit your needs, and > there is a free trial! Depending on how your work project is > oriented, it may be of interest. > > Best regards, Ryan > > On Fri, Oct 11, 2019 at 12:26 PM Steve973 <[email protected]> wrote: > > > > Thank you for your reply. I will check it out! I'm in the evaluation > > phase, especially since I have some time before I have to implement all of > > this. > > > > On Fri, Oct 11, 2019 at 3:25 AM Dan <[email protected]> wrote: > >> > >> I'm not sure if this will help but kettle runs on beam too. > >> > >> https://github.com/mattcasters/kettle-beam > >> > >> https://youtu.be/vgpGrQJnqkM > >> > >> Depends on your use case but kettle rocks for etl. > >> > >> Dan > >> > >> Sent from my phone > >> > >> On Thu, 10 Oct 2019, 10:12 pm Steve973, <[email protected]> wrote: > >>> > >>> Hello, all. I still have not been given the tasking to convert my work > >>> project to use Beam, but it is still something that I am looking to do in > >>> the fairly near future. Our data workflow consists of ingest and > >>> transformation, and I was hoping that there are ETL frameworks that work > >>> well with Beam. Does anyone have some recommendations and maybe some > >>> samples that show how people might use and ETL framework with Beam? > >>> > >>> Thanks in advance and have a great day!
