Re: [Pyspark] How to download Zip file from SFTP location and put in into Azure Data Lake and unzip it

[Wes Peng]

How large is the file? From my experience, reading the excel file from 
data lake and loading as dataframe, works great.


Thanks

On 2022-01-18 22:16, Heta Desai wrote:

Hello,

 I have zip files on SFTP location. I want to download/copy those
files and put into Azure Data Lake. Once the zip files get stored into
Azure Data Lake, I want to unzip those files and read using Data
Frames.

 The file format inside zip is excel. SO, once files are unzipped, I
want to read excel files using spark DataFrames.

 Please help me with the solution as soon as possible.

 Thanks,

 ​Heta Desai | Data | Sr Associate L1
e.heta.de...@1rivet.com | t. +91 966.225.4954

 ​

 This email, including attachments, may include confidential and/or
proprietary information, and may be used only by the person or entity
to which it is addressed. If you are not the intended recipient,
please advise the sender immediately and delete this message and any
attachments. Unless otherwise specifically stated in this email,
transaction related information in this email, including attachments,
is not to be construed as an offer, solicitation or the basis or
confirmation for any contract for the purchase/sale of any services.
All email sent to or from this address will be received by 1Rivet US,
Inc and is subject to archival retention and review by someone other
than the recipient.


-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org

[Pyspark] How to download Zip file from SFTP location and put in into Azure Data Lake and unzip it

[Heta Desai]

Hello,

I have zip files on SFTP location. I want to download/copy those files and put 
into Azure Data Lake. Once the zip files get stored into Azure Data Lake, I 
want to unzip those files and read using Data Frames.

The file format inside zip is excel. SO, once files are unzipped, I want to 
read excel files using spark DataFrames.

Please help me with the solution as soon as possible.


Thanks,
[cid:1885f47f-6a84-4099-9a4f-eabe570a41ac]  ​Heta Desai | Data | Sr 
Associate L1
e.heta.de...@1rivet.com | t. +91 966.225.4954
​

This email, including attachments, may include confidential and/or proprietary 
information, and may be used only by the person or entity to which it is 
addressed. If you are not the intended recipient, please advise the sender 
immediately and delete this message and any attachments. Unless otherwise 
specifically stated in this email, transaction related information in this 
email, including attachments, is not to be construed as an offer, solicitation 
or the basis or confirmation for any contract for the purchase/sale of any 
services. All email sent to or from this address will be received by 1Rivet US, 
Inc and is subject to archival retention and review by someone other than the 
recipient.