Hi Joe, since it would actually create a huge added value for us (and certainly for many people in the community) if everything worked out the way we wanted it to, I will actively pursue the topic. We have the contacts to Databricks and will ask them what possibilities there are here.
I think this is a win-win situation for all sides. And that it is technically possible has already been proven by different vendors, including streamsets. Personally, I am not only interested in a solution with databricks, but also in one that everyone can use. All you need for Delta is Apache Spark version 2.4.2 or above. Best Joe Witt <[email protected]> schrieb am Fr., 27. März 2020, 21:42: > Martin, > > If I follow that blog post correctly Databricks (a vendor) in partnership > with several other vendors created a set of integrations to feed data into > Delta Lake. It might make sense then for Databricks to partner with the > appropriate vendor around Apache NiFi to have that same kind of > engagement/collaboration/etc.. > > Here though - in the Apache NiFi community it is just simply about the > community establishing some JIRAs and what it wants to do as it relates to > feeding data from Apache NiFi into Delta Lake and knocking it out. You seem > to have been learning a lot about NiFi for the past couple months. And I > suspect you already know a good bit about Delta Lake. You're perhaps in > the best position to help identify some things you think we should do. If > that is right I'm looking forward to hearing some ideas you have. There is > clearly something we can do here - or maybe already it works... But lets > find out. You'll likely have to help lead/push/advocate for that but with > some momentum I think you'll find folks help. > > Thanks > > On Fri, Mar 27, 2020 at 3:38 PM Martin Ebert <[email protected]> > wrote: > > > Hi community, > > are there any plans to adopt this auto loader functionality: > > > > > https://databricks.com/blog/2020/02/24/introducing-databricks-ingest-easy-data-ingestion-into-delta-lake.html > > > > What are the plans for NiFi 2020? > > > > Best, > > Martin > > >
