Re: [EXTERNAL] Re: Spark streaming - Data Ingestion

2022-08-17 Thread Akash Vellukai
; *Sent:* 17 August 2022 16:53 > *To:* Akash Vellukai > *Cc:* user@spark.apache.org > *Subject:* [EXTERNAL] Re: Spark streaming - Data Ingestion > > *Caution! This email originated outside of FedEx. Please do not open > attachments or click links from an unknown or suspicious origi

Re: [EXTERNAL] Re: Spark streaming - Data Ingestion

2022-08-17 Thread Gibson
t;> Regards >> -- >> *From:* Gibson >> *Sent:* 17 August 2022 16:53 >> *To:* Akash Vellukai >> *Cc:* user@spark.apache.org >> *Subject:* [EXTERNAL] Re: Spark streaming - Data Ingestion >> >> *Caution! This email originat

Re: [EXTERNAL] Re: Spark streaming - Data Ingestion

2022-08-17 Thread Saurabh Gulati
; Hive Regards From: Gibson Sent: 17 August 2022 16:53 To: Akash Vellukai Cc: user@spark.apache.org Subject: [EXTERNAL] Re: Spark streaming - Data Ingestion Caution! This email originated outside of FedEx. Please do not open attachments or click links from an unknown or su

Re: Spark streaming - Data Ingestion

2022-08-17 Thread Gibson
If you have space for a message log like, then you should try: MySQL -> Kafka (via CDC) -> Spark (Structured Streaming) -> HDFS/S3/ADLS -> Hive On Wed, Aug 17, 2022 at 5:40 PM Akash Vellukai wrote: > Dear sir > > I have tried a lot on this could you help me with this? > > Data ingestion from

Spark streaming - Data Ingestion

2022-08-17 Thread Akash Vellukai
Dear sir I have tried a lot on this could you help me with this? Data ingestion from MySql to Hive with spark- streaming? Could you give me an overview. Thanks and regards Akash P