IIUC Glue is a service to easily schedule spark jobs in AWS. To be clear, its not a pre-requisite for hudi. Is this your current means of scheduling spark jobs?
On Tue, Mar 5, 2019 at 10:27 AM Umesh Kacha <[email protected]> wrote: > Hi does anybody can help with steps using Hudi in AWS ecosystem if possible > using AWS Glue then it would be a really great help and start for me and I > think for others as well looking for similar help? > > Thanks in advance. > > Regards, > Umesh > > On Tue, Mar 5, 2019, 2:02 AM Vinoth Chandar <[email protected]> wrote: > > > It did need AWS to enable a way to drop the hudi jars/support custom > presto > > plugins into the provisioned cluster. > > I quickly checked the Athena docs again. Seems like this is still a gap. > :| > > > > > > > > On Mon, Mar 4, 2019 at 12:22 PM Brandon Geise <[email protected]> > > wrote: > > > > > Is it possible to get Athena working with Hudi or does it require AWS > to > > > add support directly? > > > > > > Thanks > > > Brandon > > > > > > On 3/4/19, 1:52 PM, "Vinoth Chandar" <[email protected]> wrote: > > > > > > +1 > > > > > > Hudi writer is just a spark job. We have seen folks use EMR and > Glue > > > like > > > any other spark job, write data out to s3. > > > On Presto, again I have seen it being done if you are running > Presto > > > on a > > > bunch of ec2 machines. > > > > > > We have had discussions with AWS Athena on supporting this > > out-of-box. > > > But > > > there were some blockers on their side to take it forward at that > > time. > > > > > > Thanks > > > Vinoth > > > > > > On Mon, Mar 4, 2019 at 4:54 AM Kabeer Ahmed <[email protected]> > > > wrote: > > > > > > > Hi Umesh, > > > > > > > > We use it on AWS. So it definitely works on AWS (S3). I have read > > > that > > > > there is support for Presto too but we havent used Presto due to > > > > compatibility issues between Apache Ranger and Presto. > > > > Thanks, > > > > Kabeer. > > > > > > > > On Mar 4 2019, at 10:21 am, Umesh Kacha <[email protected]> > > > wrote: > > > > > Hi is there out of the box support for Hudi inside AWS just > like > > > Presto? > > > > > > > > > > Thanks in advance. > > > > > Regards, > > > > > Umesh > > > > > > > > > > > > > > > > > > > > > > > > > > > >
