[
https://issues.apache.org/jira/browse/HUDI-2083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17374545#comment-17374545
]
ASF GitHub Bot commented on HUDI-2083:
--------------------------------------
veenaypatil opened a new pull request #3222:
URL: https://github.com/apache/hudi/pull/3222
## What is the purpose of the pull request
This change adds support to Hudi CLI to work with S3 filesystem, details of
the exception are captured here -
https://issues.apache.org/jira/browse/HUDI-2083
## Brief change log
- Updated Hudi-cli pom.xml to exclude Hadoop related jars, this is done to
make sure there are no conflicts with users Hadoop installation
- Updated Hudi-cli.sh to point to Hadoop jars present in $HADOOP_HOME
## Verify this pull request
- This change is tested on HDP with Hadoop version 2.7.3.2.6.5.0-292
- This change is also tested on Hadoop version 3.2.2
## Committer checklist
- [X] Has a corresponding JIRA in PR title & commit
- [X] Commit message is descriptive of the change
- [ ] CI is green
- [ ] Necessary doc changes done or have another open PR
- [ ] For large changes, please consider breaking it into sub-tasks under
an umbrella JIRA.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
> Hudi CLI does not work with S3
> ------------------------------
>
> Key: HUDI-2083
> URL: https://issues.apache.org/jira/browse/HUDI-2083
> Project: Apache Hudi
> Issue Type: Task
> Components: CLI
> Reporter: Vinay
> Assignee: Vinay
> Priority: Major
>
> Hudi CLI gives exception when trying to connect to s3 path
> {code:java}
> create --path s3://some-bucket/tmp/hudi/test_mor --tableName test_mor_s3
> --tableType MERGE_ON_READ
> Failed to get instance of org.apache.hadoop.fs.FileSystem
> org.apache.hudi.exception.HoodieIOException: Failed to get instance of
> org.apache.hadoop.fs.FileSystem
> at org.apache.hudi.common.fs.FSUtils.getFs(FSUtils.java:98)
> =========
> create --path s3a://some-bucket/tmp/hudi/test_mor --tableName test_mor_s3
> --tableType MERGE_ON_READ
> Command failed java.lang.RuntimeException: java.lang.ClassNotFoundException:
> Class org.apache.hadoop.fs.s3a.S3AFileSystem not found
> java.lang.ClassNotFoundException: Class
> org.apache.hadoop.fs.s3a.S3AFileSystem not found
> java.lang.RuntimeException: java.lang.ClassNotFoundException: Class
> org.apache.hadoop.fs.s3a.S3AFileSystem not found
> at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2195)
> at
> org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2654)
> {code}
> This could be because target/lib folder does not contain hadoop-aws or aws-s3
> dependency.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
