Opened a JIRA for this https://issues.apache.org/jira/browse/HUDI-95
@Semantic Beeng <[email protected]> & I are also interested in this. Brandon, just wondering if you made any strides on this? Happy to work with you on the JIRA if you are still interested On Mon, Mar 18, 2019 at 2:00 PM nishith agarwal <[email protected]> wrote: > Brandon, > > This sounds great! We've also thought around similar lines of hot % vs cold > % all accessible through a unified query interface. Let us know how your > POC turns out. > > -Nishith > > On Mon, Mar 18, 2019 at 1:04 PM Brandon Geise <[email protected]> > wrote: > > > Will do on the JIRA. I need to do a POC to see if it's something to > > pursue on our side. > > > > Our use case would be similar using Spark + Presto and then having a 30 > > day hot storage and cold being on S3 and was thinking of having Alluxio > be > > cache/memory portion. > > > > On 3/18/19, 3:49 PM, "Vinoth Chandar" <[email protected]> wrote: > > > > Great! I can definitely see use-cases for refreshing an > alluxio/ignite > > cache incrementally.. > > > > i.e Hive => ETL => Hudi on DFS => Incremental Pull + upsert => Hudi > on > > in > > memory FS > > > > > > if you want to pursue a JIRA, please let me know. I will add you as a > > contributor > > > > On Mon, Mar 18, 2019 at 12:41 PM Brandon Geise < > [email protected] > > > > > wrote: > > > > > Thanks, Vinoth. I'll take a look. It seems to be a great starting > > point! > > > > > > On 3/18/19, 3:35 PM, "Vinoth Chandar" <[email protected]> wrote: > > > > > > I have actually played around with Apache Ignite integration > > which > > > supports > > > append(). > > > > > > > > > https://github.com/vinothchandar/incubator-hudi/commit/dd578947ec1db9388038f0a1863a90b3761cd571 > > > > > > > > > Alluxio would work as well I believe > > > > > > Something like Kafka => DeltaStreamer => Hudi/igfs could give > > you a > > > mutable > > > in-memory near real time analytics > > > (sorry for bundling up so many buzzwords :P) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Mar 18, 2019 at 12:11 PM Brandon Geise < > > [email protected] > > > > > > > wrote: > > > > > > > Hi, > > > > > > > > > > > > > > > > Has anyone used Hudi in combination with Alluxio? Based on > my > > > > understanding of each solution, it seems that at a file level > > this > > > > could/should all work together, but if someone has direct > > experience > > > I’d > > > > love to hear about it. > > > > > > > > > > > > > > > > Thanks, > > > > > > > > Brandon > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
