Hi all, I have data in S3 bucket in different folders and want to copy to HDFS directly . I used Distcp to copy files to HDFS by specifying complete s3 path of the file. I have 2 queries :
1. Can I copy the entire folder from S3 to HDFS using distcp. 2. Can I generate the exact input folder replica on HDFS as destination folder. I tried using -f <uri_list> but in that also, I need to provide the complete path of the S3 files. Please suggest Stuti ::DISCLAIMER:: ---------------------------------------------------------------------------------------------------------------------------------------------------- The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents (with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates. Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of authorized representative of HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any email and/or attachments, please check them for viruses and other defects. ----------------------------------------------------------------------------------------------------------------------------------------------------
