RE: running Hudi in AWS Glue Spark

2020-03-09 Thread Sanchez, Jorge
, 2020 7:03 AM To: dev@hudi.apache.org Subject: Re: running Hudi in AWS Glue Spark EXTERNAL EMAIL – Use caution with any links or file attachments. I actually understood more about your use-case also now, Raymond! thanks for the response! On Fri, Mar 6, 2020 at 7:02 PM Shiyan Xu wrote: > I

Re: running Hudi in AWS Glue Spark

2020-03-09 Thread Vinoth Chandar
gt; > I understand that these question should be addressed to the AWS guys, > > hoping that there are some of them on this channel. > > > > Regards, > > > > Jorge > > > > -Original Message- > > From: Vinoth Chandar > > Sent: Friday,

Re: running Hudi in AWS Glue Spark

2020-03-06 Thread Mehrotra, Udit
Hi Jorge, AWS Glue service itself does not support Hudi. However, you can use Glue as a metastore with Hudi on EMR. Hope that answers your question. Thanks, Udit Mehrotra SDE | AWS EMR On 3/6/20, 9:44 AM, "Vinoth Chandar" wrote: CAUTION: This email originated from outside of the

Re: running Hudi in AWS Glue Spark

2020-03-06 Thread Shiyan Xu
. > > Regards, > > Jorge > > -Original Message- > From: Vinoth Chandar > Sent: Friday, March 6, 2020 6:43 PM > To: dev@hudi.apache.org > Subject: Re: running Hudi in AWS Glue Spark > > EXTERNAL EMAIL – Use caution with any links or file attachments.

RE: running Hudi in AWS Glue Spark

2020-03-06 Thread Sanchez, Jorge
in the future I understand that these question should be addressed to the AWS guys, hoping that there are some of them on this channel. Regards, Jorge -Original Message- From: Vinoth Chandar Sent: Friday, March 6, 2020 6:43 PM To: dev@hudi.apache.org Subject: Re: running Hudi in AWS

Re: running Hudi in AWS Glue Spark

2020-03-06 Thread Vinoth Chandar
https://aws.amazon.com/emr/features/hudi/ mentions that its integrated with the glue catalog. It should be similar to other datasources you use on Glue IIUC.. I have seen users talk about this on slack (IIRC).. Are you running into specific issues we can help with? May be the AWS folks here can

running Hudi in AWS Glue Spark

2020-03-06 Thread Sanchez, Jorge
Hello, Did anybody tried to run Hudi within AWS Glue job, I searched the JIRA issues but did not find anybody mentioning that. Thanks, Jorge Notice: This e-mail message, together with any attachments, contains information of Merck & Co., Inc. (2000 Galloping Hill Road, Kenilworth, New