GitHub user sujith71955 opened a pull request:
https://github.com/apache/carbondata/pull/2366
[CARBONDATA-2532][Integration] Carbon to support spark 2.3 version
```
## What changes were proposed in this pull request?
In this PR inorder to hide the compatibility issues of columnar vector
API's from the existing common classes, i introduced an interface of the proxy
vector readers, this
proxy vector readers will take care the compatibility issues with respect
to spark different versions.
Column vector and Columnar Batch interface compatibility issues has been
addressed in this PR, The changes were related to below modifications done in
spark interface.
Highlights:
a) This is a refactoring of ColumnVector hierarchy and related classes.
b) make ColumnVector read-only
c) introduce WritableColumnVector with write interface
d) remove ReadOnlyColumnVector
e) Fixed spark-carbon integration API compatibility issues - By Sandi
f) Corrected the testcases based on spark 2.3.0 behaviour change
g) Excluded following dependency from pom.xml files
<groupId>net.jpountz</groupId><artifactId>lz4</artifactId> as spark 2.3.0
changed
it to org.lz4, so removed from the test class path of
spark2,spark-common-test,spark2-examples
## How was this patch tested?
Manual testing, and existing test-case execution
```
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/sujith71955/incubator-carbondata
spark-2.3_carbon_spark_2.3
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/carbondata/pull/2366.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2366
----
commit 9d68b4270c46b99d8d7985069ce633a60c04ba87
Author: sujith71955 <sujithchacko.2010@...>
Date: 2018-05-24T05:51:50Z
[CARBONDATA-2532][Integration] Carbon to support spark 2.3 version
## What changes were proposed in this pull request?
Column vector and Columnar Batch interface compatibility issues has been
addressed in this PR, The changes were related to below modifications done in
spark interface
a) This is a refactoring of ColumnVector hierarchy and related classes.
b) make ColumnVector read-only
c) introduce WritableColumnVector with write interface
d) remove ReadOnlyColumnVector
In this PR inorder to hide the compatibility issues of columnar vector
API's from the existing common classes, i introduced an interface of the proxy
vector readers, this
proxy vector readers will take care the compatibility issues with respect
to spark different versions.
## How was this patch tested?
Manual testing, and existing test-case execution
[CARBONDATA-2532][Integration] Carbon to support spark 2.3 version
## What changes were proposed in this pull request?
Column vector and Columnar Batch interface compatibility issues has been
addressed in this PR, The changes were related to below modifications done in
spark interface
a) This is a refactoring of ColumnVector hierarchy and related classes.
b) make ColumnVector read-only
c) introduce WritableColumnVector with write interface
d) remove ReadOnlyColumnVector
In this PR inorder to hide the compatibility issues of columnar vector
API's from the existing common classes, i introduced an interface of the proxy
vector readers, this
proxy vector readers will take care the compatibility issues with respect
to spark different versions.
## How was this patch tested?
Manual testing, and existing test-case execution
commit 8e62ac800c39ee308466f70d574ce1892dde9436
Author: sujith71955 <sujithchacko.2010@...>
Date: 2018-06-08T07:27:39Z
```
## What changes were proposed in this pull request?
In this PR inorder to hide the compatibility issues of columnar vector
API's from the existing common classes, i introduced an interface of the proxy
vector readers, this
proxy vector readers will take care the compatibility issues with respect
to spark different versions.
Column vector and Columnar Batch interface compatibility issues has been
addressed in this PR, The changes were related to below modifications done in
spark interface.
Highlights:
a) This is a refactoring of ColumnVector hierarchy and related classes.
b) make ColumnVector read-only
c) introduce WritableColumnVector with write interface
d) remove ReadOnlyColumnVector
e) Fixed spark-carbon integration API compatibility issues - By Sandi
f) Corrected the testcases based on spark 2.3.0 behaviour change
g) Excluded following dependency from pom.xml files
<groupId>net.jpountz</groupId><artifactId>lz4</artifactId> as spark 2.3.0
changed
it to org.lz4, so removed from the test class path of
spark2,spark-common-test,spark2-examples
## How was this patch tested?
Manual testing, and existing test-case execution
```
commit 233f1ce3e667f5ca9c7d874bf3d4dc216cb484ef
Author: sandeep-katta <sandeep.katta2007@...>
Date: 2018-05-25T10:17:48Z
WIP changed to completed.Only 1 testcase related to projection is failing.
Carbon support for 2.3.1
commit b571400710df5fb0229870a2b0e7d73bfc9995af
Author: sandeep-katta <sandeep.katta2007@...>
Date: 2018-06-08T11:18:22Z
Carbon need to support spark 2.3.0 version
----
---
