GitHub user sujith71955 opened a pull request:
https://github.com/apache/carbondata/pull/2364
[CARBONDATA-2592][Integration] Getting NoSuchMethod error due to aws sdk
multple version jar conflicts
```
## What changes were proposed in this pull request?
Currently in Carbon Spark2 project multiple dependency for the aws-sdk jar
has been defined,this will create issue when we build carbon-assembly jars with
latest versions of hadoop/spark project. As part of hadoop-aws project,
already aws-sdk 1.10.6 version jar will be fetched, since in the carbon-spark2
pom.xml
there is an explicit dependency defined/hardcoded for aws-sdk 1.7.4(old
version) this can lead to conflicts while loading the class files. because of
this problem
when we run any carbon examples passing carbon-assembly jars as the
argument using spark-submit none of the testcases will work.
As a solution we can remove this dependency(aws-sdk 1.7.4) as already
hadoop-aws dependency defined in the pom.xml of carbon-spark2 project
will fetch the latest aws-sdk jars.
## How was this patch tested?
After updating the pom, manually projects has been build and the use-case
mentioned above is manually tested.```
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/sujith71955/incubator-carbondata master_pom
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/carbondata/pull/2364.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2364
----
commit 6fe46836fd72e2efa5e1b24742601439f6d71fb0
Author: sujith71955 <sujithchacko.2010@...>
Date: 2018-06-07T07:08:56Z
[CARBONDATA-2592][Integration] Getting NoSuchMethod error due to aws-sdk
multple version jar conflicts
## What changes were proposed in this pull request?
Currently in Carbon Spark2 project multiple dependency for the aws-sdk jar
has been defined,this will create issue when we build carbon-assembly jars with
latest versions of hadoop/spark project. As part of hadoop-aws project,
already aws-sdk 1.10.6 version jar will be fetched, since in the carbon-spark2
pom.xml
there is an explicit dependency defined/hardcoded for aws-sdk 1.7.4(old
version) this can lead to conflicts while loading the class files. because of
this problem
when we run any carbon examples passing carbon-assembly jars as the
argument using spark-submit none of the testcases will work.
As a solution we can remove this dependency(aws-sdk 1.7.4) as already
hadoop-aws dependency defined in the pom.xml of carbon-spark2 project
will fetch the latest aws-sdk jars.
## How was this patch tested?
After updating the pom, manually projects has been build and the use-case
mentioned above is manually tested.
## How was this patch tested?
----
---
