No problem :). On Tue, Apr 9, 2019 at 10:31 AM Brandon Geise <[email protected]> wrote:
> Hi Vinoth, > > I haven't. As a company we started leaning towards ignite so we can focus > on a cache solution that supports multiple use cases. I am still > interested in spending time on this as time allows on my side, but > currently can't invest as much time as I originally thought. > > On 4/9/19, 1:28 PM, "Vinoth Chandar" <[email protected]> wrote: > > Opened a JIRA for this https://issues.apache.org/jira/browse/HUDI-95 > > @Semantic Beeng <[email protected]> & I are also interested in > this. > Brandon, just wondering if you made any strides on this? > Happy to work with you on the JIRA if you are still interested > > On Mon, Mar 18, 2019 at 2:00 PM nishith agarwal <[email protected]> > wrote: > > > Brandon, > > > > This sounds great! We've also thought around similar lines of hot % > vs cold > > % all accessible through a unified query interface. Let us know how > your > > POC turns out. > > > > -Nishith > > > > On Mon, Mar 18, 2019 at 1:04 PM Brandon Geise < > [email protected]> > > wrote: > > > > > Will do on the JIRA. I need to do a POC to see if it's something > to > > > pursue on our side. > > > > > > Our use case would be similar using Spark + Presto and then having > a 30 > > > day hot storage and cold being on S3 and was thinking of having > Alluxio > > be > > > cache/memory portion. > > > > > > On 3/18/19, 3:49 PM, "Vinoth Chandar" <[email protected]> wrote: > > > > > > Great! I can definitely see use-cases for refreshing an > > alluxio/ignite > > > cache incrementally.. > > > > > > i.e Hive => ETL => Hudi on DFS => Incremental Pull + upsert => > Hudi > > on > > > in > > > memory FS > > > > > > > > > if you want to pursue a JIRA, please let me know. I will add > you as a > > > contributor > > > > > > On Mon, Mar 18, 2019 at 12:41 PM Brandon Geise < > > [email protected] > > > > > > > wrote: > > > > > > > Thanks, Vinoth. I'll take a look. It seems to be a great > starting > > > point! > > > > > > > > On 3/18/19, 3:35 PM, "Vinoth Chandar" <[email protected]> > wrote: > > > > > > > > I have actually played around with Apache Ignite > integration > > > which > > > > supports > > > > append(). > > > > > > > > > > > > > > https://github.com/vinothchandar/incubator-hudi/commit/dd578947ec1db9388038f0a1863a90b3761cd571 > > > > > > > > > > > > Alluxio would work as well I believe > > > > > > > > Something like Kafka => DeltaStreamer => Hudi/igfs could > give > > > you a > > > > mutable > > > > in-memory near real time analytics > > > > (sorry for bundling up so many buzzwords :P) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Mar 18, 2019 at 12:11 PM Brandon Geise < > > > [email protected] > > > > > > > > > wrote: > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > Has anyone used Hudi in combination with Alluxio? > Based on > > my > > > > > understanding of each solution, it seems that at a > file level > > > this > > > > > could/should all work together, but if someone has > direct > > > experience > > > > I’d > > > > > love to hear about it. > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > Brandon > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
