Something like this? https://ignite.apache.org/use-cases/hadoop-acceleration.html
On Thu., Sep. 22, 2022, 3:44 a.m. Stephen Darlington, < [email protected]> wrote: > I don’t know of anyone doing this, however it looks like it should be > possible. > > According to a quick skim of the docs, to read/write to Hudl you need > Flink or Spark. To use the Cache Store (read/write-through) you’d need to > embed one of those inside Ignite, so plenty of opportunity for “dependency > hell.” I do know of one project where they embedded Spark. > > On 22 Sep 2022, at 03:58, Tecno Brain <[email protected]> > wrote: > > I have heard of a tool called Alluxio used between Hudi and Spark/Presto. ( > https://www.alluxio.io/blog/building-high-performance-data-lake-using-apache-hudi-and-alluxio-at-t3go/ > ) > I was wondering if Apache Ignite could serve the same purpose, allowing > queries to be processed faster. > > On Thu, Sep 15, 2022 at 10:29 AM Jeremy McMillan < > [email protected]> wrote: > >> I just read this, about hudi, and I can't see a use case for putting hudi >> behind an Ignite write-through cache. >> >> https://www.xenonstack.com/insights/what-is-hudi >> >> Hudi seems to be a write accelerator for Spark on HDFS, primarily. >> >> What would the expected outcome be if we assume the magic integration was >> present and working as you intend? What's the difference between that and >> not using Ignite with Hudi? >> >> On Wed, Sep 14, 2022, 22:50 Tecno Brain <[email protected]> >> wrote: >> >>> In particular I am looking if anyone has used Apache Ignite as a >>> write-through cache to Hudi. >>> Does that make sense? >>> >>> On Wed, Sep 14, 2022 at 10:50 PM Tecno Brain < >>> [email protected]> wrote: >>> >>>> I was wondering if anybody has used Hudi + Ignite? >>>> Any references to articles, conferences are greatly appreciated. >>>> >>>> Thanks >>>> >>>> >>>> >>>> >
